2023-03-03T06:07:54.948 INFO:root:teuthology version: 0.0.1.dev49+gc7c3cc8 2023-03-03T06:07:54.949 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --archive /home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192143 --owner scheduled_vshankar@teuthology --verbose --name vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi -- /home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192143/orig.config.yaml 2023-03-03T06:07:55.081 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-03-03T06:07:55.155 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192143 branch: wip-vshankar-testing-20230303.023823 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: vshankar@redhat.com first_in_suite: false job_id: '7192143' kernel: client: branch: testing flavor: default kdb: true sha1: 75534f74ac3dd06f86697ff862e8be2945b3c8e6 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-vshankar-testing-20230303.023823 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 472aaef4ec3b3a2c29f73ca67854785a910f1f9b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 472aaef4ec3b3a2c29f73ca67854785a910f1f9b kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20230303.023823 sha1: 472aaef4ec3b3a2c29f73ca67854785a910f1f9b owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 472aaef4ec3b3a2c29f73ca67854785a910f1f9b sleep_before_teardown: 0 suite: fs suite_branch: wip-vshankar-testing-20230303.023823 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_472aaef4ec3b3a2c29f73ca67854785a910f1f9b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 472aaef4ec3b3a2c29f73ca67854785a910f1f9b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi003.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0= smithi032.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6 timestamp: 2023-03-03_04:39:14 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.31660 2023-03-03T06:07:55.156 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_472aaef4ec3b3a2c29f73ca67854785a910f1f9b/qa; will attempt to use it 2023-03-03T06:07:55.157 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_472aaef4ec3b3a2c29f73ca67854785a910f1f9b/qa/tasks 2023-03-03T06:07:55.157 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-03-03T06:07:55.161 INFO:teuthology.task.internal:Checking packages... 2023-03-03T06:07:55.197 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '472aaef4ec3b3a2c29f73ca67854785a910f1f9b' 2023-03-03T06:07:55.197 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-03-03T06:07:55.198 INFO:teuthology.packaging:ref: None 2023-03-03T06:07:55.198 INFO:teuthology.packaging:tag: None 2023-03-03T06:07:55.198 INFO:teuthology.packaging:branch: wip-vshankar-testing-20230303.023823 2023-03-03T06:07:55.198 INFO:teuthology.packaging:sha1: 472aaef4ec3b3a2c29f73ca67854785a910f1f9b 2023-03-03T06:07:55.199 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-vshankar-testing-20230303.023823 2023-03-03T06:07:55.342 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2739-g472aaef4-1focal 2023-03-03T06:07:55.343 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-03-03T06:07:55.385 INFO:teuthology.task.internal:no buildpackages task found 2023-03-03T06:07:55.386 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-03-03T06:07:55.392 INFO:teuthology.task.internal:Saving configuration 2023-03-03T06:07:55.410 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-03-03T06:07:55.418 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-03-03T06:07:55.449 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192143', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-03 06:02:01.920103', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-03-03T06:07:55.473 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192143', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-03 06:02:01.921827', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-03-03T06:07:55.473 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-03-03T06:07:55.487 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-03-03T06:07:55.487 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-03-03T06:07:55.487 INFO:teuthology.run_tasks:Running task console_log... 2023-03-03T06:07:55.567 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8b821915e0>, signals=[15]) 2023-03-03T06:07:55.568 INFO:teuthology.run_tasks:Running task internal.connect... 2023-03-03T06:07:55.587 INFO:teuthology.task.internal:Opening connections... 2023-03-03T06:07:55.587 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2023-03-03T06:07:55.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-03T06:07:55.643 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2023-03-03T06:07:55.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-03T06:07:55.698 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-03-03T06:07:55.720 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2023-03-03T06:07:55.727 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2023-03-03T06:07:55.728 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2023-03-03T06:07:55.773 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Ubuntu" 2023-03-03T06:07:55.774 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-03-03T06:07:55.774 INFO:teuthology.orchestra.run.smithi003.stdout:ID=ubuntu 2023-03-03T06:07:55.774 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE=debian 2023-03-03T06:07:55.774 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-03-03T06:07:55.774 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="20.04" 2023-03-03T06:07:55.775 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.ubuntu.com/" 2023-03-03T06:07:55.775 INFO:teuthology.orchestra.run.smithi003.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-03-03T06:07:55.775 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-03-03T06:07:55.775 INFO:teuthology.orchestra.run.smithi003.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-03-03T06:07:55.775 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_CODENAME=focal 2023-03-03T06:07:55.775 INFO:teuthology.orchestra.run.smithi003.stdout:UBUNTU_CODENAME=focal 2023-03-03T06:07:55.776 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2023-03-03T06:07:55.797 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2023-03-03T06:07:55.804 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2023-03-03T06:07:55.805 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-03-03T06:07:55.853 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Ubuntu" 2023-03-03T06:07:55.854 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-03-03T06:07:55.854 INFO:teuthology.orchestra.run.smithi032.stdout:ID=ubuntu 2023-03-03T06:07:55.854 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE=debian 2023-03-03T06:07:55.854 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-03-03T06:07:55.854 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="20.04" 2023-03-03T06:07:55.854 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.ubuntu.com/" 2023-03-03T06:07:55.854 INFO:teuthology.orchestra.run.smithi032.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-03-03T06:07:55.855 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-03-03T06:07:55.855 INFO:teuthology.orchestra.run.smithi032.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-03-03T06:07:55.855 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_CODENAME=focal 2023-03-03T06:07:55.855 INFO:teuthology.orchestra.run.smithi032.stdout:UBUNTU_CODENAME=focal 2023-03-03T06:07:55.855 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-03-03T06:07:55.880 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-03-03T06:07:55.928 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-03-03T06:07:55.941 INFO:teuthology.task.internal:Checking for old test directory... 2023-03-03T06:07:55.942 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2023-03-03T06:07:55.945 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2023-03-03T06:07:55.950 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-03-03T06:07:55.993 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-03-03T06:07:55.994 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2023-03-03T06:07:55.996 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2023-03-03T06:07:56.017 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-03-03T06:07:56.095 INFO:teuthology.run_tasks:Running task kernel... 2023-03-03T06:07:56.137 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '75534f74ac3dd06f86697ff862e8be2945b3c8e6'} 2023-03-03T06:07:56.137 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-03-03T06:07:56.138 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-03-03T06:07:56.138 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-03T06:07:56.139 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-03T06:07:56.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-03T06:07:56.145 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=testing 2023-03-03T06:07:56.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-03T06:07:56.149 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=testing 2023-03-03T06:07:56.357 DEBUG:teuthology.task.kernel:sha1 for client.0 is 75534f74ac3dd06f86697ff862e8be2945b3c8e6 2023-03-03T06:07:56.357 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "75534f74ac3dd06f86697ff862e8be2945b3c8e6"... 2023-03-03T06:07:56.358 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2023-03-03T06:07:56.364 INFO:teuthology.orchestra.run.smithi003.stdout:5.4.0-124-generic 2023-03-03T06:07:56.365 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs 75534f74ac3dd06f86697ff862e8be2945b3c8e6 2023-03-03T06:07:56.365 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-03-03T06:07:56.366 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2023-03-03T06:07:56.366 DEBUG:teuthology.orchestra.run.smithi003:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-03-03T06:07:56.383 DEBUG:teuthology.task.kernel:sha1 for client.1 is 75534f74ac3dd06f86697ff862e8be2945b3c8e6 2023-03-03T06:07:56.383 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "75534f74ac3dd06f86697ff862e8be2945b3c8e6"... 2023-03-03T06:07:56.383 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-03-03T06:07:56.388 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2023-03-03T06:07:56.390 INFO:teuthology.orchestra.run.smithi032.stdout:5.4.0-124-generic 2023-03-03T06:07:56.390 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs 75534f74ac3dd06f86697ff862e8be2945b3c8e6 2023-03-03T06:07:56.390 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-03-03T06:07:56.391 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2023-03-03T06:07:56.391 DEBUG:teuthology.orchestra.run.smithi032:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-03-03T06:07:56.418 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2023-03-03T06:07:56.491 INFO:teuthology.orchestra.run.smithi003.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-03-03T06:07:56.493 DEBUG:teuthology.orchestra.run.smithi003:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-03-03T06:07:56.524 DEBUG:teuthology.orchestra.run.smithi003:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-03-03T06:07:56.528 INFO:teuthology.orchestra.run.smithi032.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-03-03T06:07:56.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-03-03T06:07:56.543 DEBUG:teuthology.orchestra.run.smithi003:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-03-03T06:07:56.604 DEBUG:teuthology.orchestra.run.smithi032:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-03-03T06:07:56.620 DEBUG:teuthology.orchestra.run.smithi032:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-03-03T06:08:59.867 INFO:teuthology.orchestra.run.smithi003.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-03-03T06:08:59.867 INFO:teuthology.orchestra.run.smithi003.stderr: * [new branch] main -> origin/main 2023-03-03T06:08:59.918 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190312 -> 20190312 2023-03-03T06:08:59.919 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190416 -> 20190416 2023-03-03T06:08:59.919 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190514 -> 20190514 2023-03-03T06:08:59.919 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190618 -> 20190618 2023-03-03T06:08:59.919 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190717 -> 20190717 2023-03-03T06:08:59.919 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190815 -> 20190815 2023-03-03T06:08:59.919 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20190923 -> 20190923 2023-03-03T06:08:59.921 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20191022 -> 20191022 2023-03-03T06:08:59.921 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20191215 -> 20191215 2023-03-03T06:08:59.921 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200122 -> 20200122 2023-03-03T06:08:59.921 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200316 -> 20200316 2023-03-03T06:08:59.922 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200421 -> 20200421 2023-03-03T06:08:59.922 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200519 -> 20200519 2023-03-03T06:08:59.922 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200619 -> 20200619 2023-03-03T06:08:59.922 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200721 -> 20200721 2023-03-03T06:08:59.922 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200817 -> 20200817 2023-03-03T06:08:59.922 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20200918 -> 20200918 2023-03-03T06:08:59.922 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20201022 -> 20201022 2023-03-03T06:08:59.923 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20201118 -> 20201118 2023-03-03T06:08:59.923 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20201218 -> 20201218 2023-03-03T06:08:59.923 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20210208 -> 20210208 2023-03-03T06:08:59.923 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20210315 -> 20210315 2023-03-03T06:08:59.923 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20210511 -> 20210511 2023-03-03T06:08:59.923 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20210716 -> 20210716 2023-03-03T06:08:59.923 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20210818 -> 20210818 2023-03-03T06:08:59.925 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20210919 -> 20210919 2023-03-03T06:08:59.925 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20211027 -> 20211027 2023-03-03T06:08:59.925 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20211216 -> 20211216 2023-03-03T06:08:59.925 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220209 -> 20220209 2023-03-03T06:08:59.925 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220310 -> 20220310 2023-03-03T06:08:59.926 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220411 -> 20220411 2023-03-03T06:08:59.926 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220509 -> 20220509 2023-03-03T06:08:59.926 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220610 -> 20220610 2023-03-03T06:08:59.926 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220708 -> 20220708 2023-03-03T06:08:59.926 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220815 -> 20220815 2023-03-03T06:08:59.926 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20220913 -> 20220913 2023-03-03T06:08:59.926 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20221012 -> 20221012 2023-03-03T06:08:59.927 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20221109 -> 20221109 2023-03-03T06:08:59.927 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20221214 -> 20221214 2023-03-03T06:08:59.927 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20230117 -> 20230117 2023-03-03T06:08:59.927 INFO:teuthology.orchestra.run.smithi003.stderr: * [new tag] 20230210 -> 20230210 2023-03-03T06:09:00.426 INFO:teuthology.orchestra.run.smithi032.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-03-03T06:09:00.427 INFO:teuthology.orchestra.run.smithi032.stderr: * [new branch] main -> origin/main 2023-03-03T06:09:00.469 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190312 -> 20190312 2023-03-03T06:09:00.470 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190416 -> 20190416 2023-03-03T06:09:00.470 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190514 -> 20190514 2023-03-03T06:09:00.470 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190618 -> 20190618 2023-03-03T06:09:00.470 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190717 -> 20190717 2023-03-03T06:09:00.470 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190815 -> 20190815 2023-03-03T06:09:00.470 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190923 -> 20190923 2023-03-03T06:09:00.471 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20191022 -> 20191022 2023-03-03T06:09:00.471 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20191215 -> 20191215 2023-03-03T06:09:00.471 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200122 -> 20200122 2023-03-03T06:09:00.471 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200316 -> 20200316 2023-03-03T06:09:00.473 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200421 -> 20200421 2023-03-03T06:09:00.473 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200519 -> 20200519 2023-03-03T06:09:00.474 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200619 -> 20200619 2023-03-03T06:09:00.474 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200721 -> 20200721 2023-03-03T06:09:00.474 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200817 -> 20200817 2023-03-03T06:09:00.474 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200918 -> 20200918 2023-03-03T06:09:00.474 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20201022 -> 20201022 2023-03-03T06:09:00.474 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20201118 -> 20201118 2023-03-03T06:09:00.474 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20201218 -> 20201218 2023-03-03T06:09:00.475 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210208 -> 20210208 2023-03-03T06:09:00.475 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210315 -> 20210315 2023-03-03T06:09:00.475 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210511 -> 20210511 2023-03-03T06:09:00.475 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210716 -> 20210716 2023-03-03T06:09:00.475 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210818 -> 20210818 2023-03-03T06:09:00.475 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210919 -> 20210919 2023-03-03T06:09:00.475 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20211027 -> 20211027 2023-03-03T06:09:00.476 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20211216 -> 20211216 2023-03-03T06:09:00.476 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220209 -> 20220209 2023-03-03T06:09:00.476 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220310 -> 20220310 2023-03-03T06:09:00.476 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220411 -> 20220411 2023-03-03T06:09:00.476 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220509 -> 20220509 2023-03-03T06:09:00.476 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220610 -> 20220610 2023-03-03T06:09:00.476 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220708 -> 20220708 2023-03-03T06:09:00.476 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220815 -> 20220815 2023-03-03T06:09:00.477 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220913 -> 20220913 2023-03-03T06:09:00.477 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20221012 -> 20221012 2023-03-03T06:09:00.477 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20221109 -> 20221109 2023-03-03T06:09:00.477 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20221214 -> 20221214 2023-03-03T06:09:00.477 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230117 -> 20230117 2023-03-03T06:09:00.477 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230210 -> 20230210 2023-03-03T06:09:03.134 INFO:teuthology.orchestra.run.smithi003.stderr:Updating files: 50% (1433/2860) Updating files: 51% (1459/2860) Updating files: 52% (1488/2860) Updating files: 53% (1516/2860) Updating files: 54% (1545/2860) Updating files: 55% (1573/2860) Updating files: 56% (1602/2860) Updating files: 57% (1631/2860) Updating files: 58% (1659/2860) Updating files: 59% (1688/2860) Updating files: 60% (1716/2860) Updating files: 61% (1745/2860) Updating files: 62% (1774/2860) Updating files: 63% (1802/2860) Updating files: 64% (1831/2860) Updating files: 64% (1857/2860) Updating files: 65% (1859/2860) Updating files: 66% (1888/2860) Updating files: 67% (1917/2860) Updating files: 68% (1945/2860) Updating files: 69% (1974/2860) Updating files: 70% (2002/2860) Updating files: 71% (2031/2860) Updating files: 72% (2060/2860) Updating files: 73% (2088/2860) Updating files: 74% (2117/2860) Updating files: 75% (2145/2860) Updating files: 76% (2174/2860) Updating files: 77% (2203/2860) Updating files: 78% (2231/2860) Updating files: 79% (2260/2860) Updating files: 80% (2288/2860) Updating files: 81% (2317/2860) Updating files: 82% (2346/2860) Updating files: 83% (2374/2860) Updating files: 84% (2403/2860) Updating files: 85% (2431/2860) Updating files: 86% (2460/2860) Updating files: 87% (2489/2860) Updating files: 88% (2517/2860) Updating files: 89% (2546/2860) Updating files: 90% (2574/2860) Updating files: 91% (2603/2860) Updating files: 92% (2632/2860) Updating files: 93% (2660/2860) Updating files: 94% (2689/2860) Updating files: 95% (2717/2860) Updating files: 96% (2746/2860) Updating files: 97% (2775/2860) Updating files: 98% (2803/2860) Updating files: 99% (2832/2860) Updating files: 100% (2860/2860) Updating files: 100% (2860/2860), done. 2023-03-03T06:09:03.139 INFO:teuthology.orchestra.run.smithi003.stdout:HEAD is now at 60971a6 Merge branch 'liquidio-20230222' of https://github.com/dchickles/linux-firmware 2023-03-03T06:09:03.153 INFO:teuthology.task.kernel:Downloading kernel 75534f74ac3dd06f86697ff862e8be2945b3c8e6 on client.0... 2023-03-03T06:09:03.154 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=75534f74ac3dd06f86697ff862e8be2945b3c8e6 2023-03-03T06:09:03.298 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/75534f74ac3dd06f86697ff862e8be2945b3c8e6/ubuntu/focal/flavors/default/pool/main/l/linux-6.2.0-ceph-g75534f74ac3d/ 2023-03-03T06:09:03.298 DEBUG:teuthology.orchestra.run.smithi003:> rm -f /tmp/linux-image.deb && echo linux-image-6.2.0-ceph-g75534f74ac3d_6.2.0-ceph-g75534f74ac3d-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/75534f74ac3dd06f86697ff862e8be2945b3c8e6/ubuntu/focal/flavors/default/pool/main/l/linux-6.2.0-ceph-g75534f74ac3d/ --input-file=- 2023-03-03T06:09:03.301 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi003.front.sepia.ceph.com to complete... 2023-03-03T06:09:03.680 INFO:teuthology.orchestra.run.smithi032.stderr:Updating files: 50% (1433/2860) Updating files: 51% (1459/2860) Updating files: 52% (1488/2860) Updating files: 53% (1516/2860) Updating files: 54% (1545/2860) Updating files: 55% (1573/2860) Updating files: 56% (1602/2860) Updating files: 57% (1631/2860) Updating files: 58% (1659/2860) Updating files: 59% (1688/2860) Updating files: 60% (1716/2860) Updating files: 61% (1745/2860) Updating files: 62% (1774/2860) Updating files: 63% (1802/2860) Updating files: 64% (1831/2860) Updating files: 64% (1857/2860) Updating files: 65% (1859/2860) Updating files: 66% (1888/2860) Updating files: 67% (1917/2860) Updating files: 68% (1945/2860) Updating files: 69% (1974/2860) Updating files: 70% (2002/2860) Updating files: 71% (2031/2860) Updating files: 72% (2060/2860) Updating files: 73% (2088/2860) Updating files: 74% (2117/2860) Updating files: 75% (2145/2860) Updating files: 76% (2174/2860) Updating files: 77% (2203/2860) Updating files: 78% (2231/2860) Updating files: 79% (2260/2860) Updating files: 80% (2288/2860) Updating files: 81% (2317/2860) Updating files: 82% (2346/2860) Updating files: 82% (2348/2860) Updating files: 83% (2374/2860) Updating files: 84% (2403/2860) Updating files: 85% (2431/2860) Updating files: 86% (2460/2860) Updating files: 87% (2489/2860) Updating files: 88% (2517/2860) Updating files: 89% (2546/2860) Updating files: 90% (2574/2860) Updating files: 91% (2603/2860) Updating files: 92% (2632/2860) Updating files: 93% (2660/2860) Updating files: 94% (2689/2860) Updating files: 95% (2717/2860) Updating files: 96% (2746/2860) Updating files: 97% (2775/2860) Updating files: 98% (2803/2860) Updating files: 99% (2832/2860) Updating files: 100% (2860/2860) Updating files: 100% (2860/2860), done. 2023-03-03T06:09:03.684 INFO:teuthology.orchestra.run.smithi032.stdout:HEAD is now at 60971a6 Merge branch 'liquidio-20230222' of https://github.com/dchickles/linux-firmware 2023-03-03T06:09:03.699 INFO:teuthology.task.kernel:Downloading kernel 75534f74ac3dd06f86697ff862e8be2945b3c8e6 on client.1... 2023-03-03T06:09:03.699 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=75534f74ac3dd06f86697ff862e8be2945b3c8e6 2023-03-03T06:09:03.841 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/75534f74ac3dd06f86697ff862e8be2945b3c8e6/ubuntu/focal/flavors/default/pool/main/l/linux-6.2.0-ceph-g75534f74ac3d/ 2023-03-03T06:09:03.841 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /tmp/linux-image.deb && echo linux-image-6.2.0-ceph-g75534f74ac3d_6.2.0-ceph-g75534f74ac3d-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/75534f74ac3dd06f86697ff862e8be2945b3c8e6/ubuntu/focal/flavors/default/pool/main/l/linux-6.2.0-ceph-g75534f74ac3d/ --input-file=- 2023-03-03T06:09:03.844 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi032.front.sepia.ceph.com to complete... 2023-03-03T06:09:15.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03 06:09:15 URL:https://4.chacra.ceph.com/r/kernel/testing/75534f74ac3dd06f86697ff862e8be2945b3c8e6/ubuntu/focal/flavors/default/pool/main/l/linux-6.2.0-ceph-g75534f74ac3d/linux-image-6.2.0-ceph-g75534f74ac3d_6.2.0-ceph-g75534f74ac3d-1_amd64.deb [38944908/38944908] -> "/tmp/linux-image.deb" [1] 2023-03-03T06:09:15.332 INFO:teuthology.orchestra.run.smithi003.stderr:FINISHED --2023-03-03 06:09:15-- 2023-03-03T06:09:15.332 INFO:teuthology.orchestra.run.smithi003.stderr:Total wall clock time: 12s 2023-03-03T06:09:15.332 INFO:teuthology.orchestra.run.smithi003.stderr:Downloaded: 1 files, 37M in 12s (3.18 MB/s) 2023-03-03T06:09:15.333 INFO:teuthology.task.kernel:Installing kernel 75534f74ac3dd06f86697ff862e8be2945b3c8e6 on client.0... 2023-03-03T06:09:15.333 DEBUG:teuthology.orchestra.run.smithi003:> sudo dpkg -i /tmp/linux-image.deb 2023-03-03T06:09:15.608 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package linux-image-6.2.0-ceph-g75534f74ac3d. 2023-03-03T06:09:17.322 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... 104373 files and directories currently installed.) 2023-03-03T06:09:17.327 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-03-03T06:09:17.446 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-image-6.2.0-ceph-g75534f74ac3d (6.2.0-ceph-g75534f74ac3d-1) ... 2023-03-03T06:09:20.467 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-image-6.2.0-ceph-g75534f74ac3d (6.2.0-ceph-g75534f74ac3d-1) ... 2023-03-03T06:09:20.581 INFO:teuthology.orchestra.run.smithi003.stderr:update-initramfs: Generating /boot/initrd.img-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:09:32.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03 06:09:32 URL:https://4.chacra.ceph.com/r/kernel/testing/75534f74ac3dd06f86697ff862e8be2945b3c8e6/ubuntu/focal/flavors/default/pool/main/l/linux-6.2.0-ceph-g75534f74ac3d/linux-image-6.2.0-ceph-g75534f74ac3d_6.2.0-ceph-g75534f74ac3d-1_amd64.deb [38944908/38944908] -> "/tmp/linux-image.deb" [1] 2023-03-03T06:09:32.592 INFO:teuthology.orchestra.run.smithi032.stderr:FINISHED --2023-03-03 06:09:32-- 2023-03-03T06:09:32.592 INFO:teuthology.orchestra.run.smithi032.stderr:Total wall clock time: 29s 2023-03-03T06:09:32.592 INFO:teuthology.orchestra.run.smithi032.stderr:Downloaded: 1 files, 37M in 28s (1.32 MB/s) 2023-03-03T06:09:32.602 INFO:teuthology.task.kernel:Installing kernel 75534f74ac3dd06f86697ff862e8be2945b3c8e6 on client.1... 2023-03-03T06:09:32.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo dpkg -i /tmp/linux-image.deb 2023-03-03T06:09:34.974 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-image-6.2.0-ceph-g75534f74ac3d. 2023-03-03T06:09:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:Sourcing file `/etc/default/grub' 2023-03-03T06:09:41.785 INFO:teuthology.orchestra.run.smithi003.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-03-03T06:09:41.810 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2023-03-03T06:09:42.124 INFO:teuthology.orchestra.run.smithi003.stderr:Found linux image: /boot/vmlinuz-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:09:42.150 INFO:teuthology.orchestra.run.smithi003.stderr:Found initrd image: /boot/initrd.img-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:09:42.554 INFO:teuthology.orchestra.run.smithi003.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-03-03T06:09:42.567 INFO:teuthology.orchestra.run.smithi003.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-03-03T06:09:44.660 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... 104373 files and directories currently installed.) 2023-03-03T06:09:44.666 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-03-03T06:09:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:done 2023-03-03T06:09:45.265 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-6.2.0-ceph-g75534f74ac3d (6.2.0-ceph-g75534f74ac3d-1) ... 2023-03-03T06:09:45.419 DEBUG:teuthology.orchestra.run.smithi003:> dpkg-deb -c /tmp/linux-image.deb 2023-03-03T06:09:45.430 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./ 2023-03-03T06:09:45.430 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./boot/ 2023-03-03T06:09:45.430 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 4431288 2023-03-03 04:49 ./boot/System.map-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:09:45.446 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 166345 2023-03-03 04:49 ./boot/config-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:09:45.446 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11098304 2023-03-03 04:49 ./boot/vmlinuz-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:09:45.467 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./etc/ 2023-03-03T06:09:45.468 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./etc/kernel/ 2023-03-03T06:09:45.468 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./etc/kernel/postinst.d/ 2023-03-03T06:09:45.468 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./etc/kernel/postrm.d/ 2023-03-03T06:09:45.468 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./etc/kernel/preinst.d/ 2023-03-03T06:09:45.468 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./etc/kernel/prerm.d/ 2023-03-03T06:09:45.468 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/ 2023-03-03T06:09:45.469 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/ 2023-03-03T06:09:45.469 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/ 2023-03-03T06:09:45.469 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/ 2023-03-03T06:09:45.469 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/ 2023-03-03T06:09:45.469 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/ 2023-03-03T06:09:45.469 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/ 2023-03-03T06:09:45.469 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 527801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/aesni-intel.ko 2023-03-03T06:09:45.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-03-03T06:09:45.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-03-03T06:09:45.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-03-03T06:09:45.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 88369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-03-03T06:09:45.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 100121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-03-03T06:09:45.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 142161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-03-03T06:09:45.472 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-03-03T06:09:45.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-03-03T06:09:45.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 78769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-03-03T06:09:45.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-03-03T06:09:45.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-03-03T06:09:45.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-03-03T06:09:45.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-03-03T06:09:45.473 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-03-03T06:09:45.474 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-03-03T06:09:45.474 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-03-03T06:09:45.474 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-03-03T06:09:45.474 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-03-03T06:09:45.474 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-03-03T06:09:45.474 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kernel/ 2023-03-03T06:09:45.474 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kernel/cpu/ 2023-03-03T06:09:45.475 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kernel/cpu/mce/ 2023-03-03T06:09:45.475 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-03-03T06:09:45.475 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kernel/cpuid.ko 2023-03-03T06:09:45.475 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kernel/msr.ko 2023-03-03T06:09:45.475 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kvm/ 2023-03-03T06:09:45.475 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 344241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kvm/kvm-amd.ko 2023-03-03T06:09:45.475 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 912137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kvm/kvm-intel.ko 2023-03-03T06:09:45.476 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 2209417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kvm/kvm.ko 2023-03-03T06:09:45.484 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/ 2023-03-03T06:09:45.484 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/af_alg.ko 2023-03-03T06:09:45.484 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/algif_hash.ko 2023-03-03T06:09:45.484 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/algif_skcipher.ko 2023-03-03T06:09:45.484 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/ansi_cprng.ko 2023-03-03T06:09:45.485 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/anubis.ko 2023-03-03T06:09:45.485 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/arc4.ko 2023-03-03T06:09:45.485 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/asymmetric_keys/ 2023-03-03T06:09:45.485 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-03-03T06:09:45.485 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/ 2023-03-03T06:09:45.485 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/async_memcpy.ko 2023-03-03T06:09:45.485 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/async_pq.ko 2023-03-03T06:09:45.485 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-03-03T06:09:45.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/async_tx.ko 2023-03-03T06:09:45.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/async_xor.ko 2023-03-03T06:09:45.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/raid6test.ko 2023-03-03T06:09:45.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/authenc.ko 2023-03-03T06:09:45.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/authencesn.ko 2023-03-03T06:09:45.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/blake2b_generic.ko 2023-03-03T06:09:45.486 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/blowfish_common.ko 2023-03-03T06:09:45.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/blowfish_generic.ko 2023-03-03T06:09:45.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/camellia_generic.ko 2023-03-03T06:09:45.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/cast5_generic.ko 2023-03-03T06:09:45.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/cast6_generic.ko 2023-03-03T06:09:45.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/cast_common.ko 2023-03-03T06:09:45.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/ccm.ko 2023-03-03T06:09:45.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/cmac.ko 2023-03-03T06:09:45.487 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/crc32_generic.ko 2023-03-03T06:09:45.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/cryptd.ko 2023-03-03T06:09:45.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/crypto_engine.ko 2023-03-03T06:09:45.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/crypto_simd.ko 2023-03-03T06:09:45.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/crypto_user.ko 2023-03-03T06:09:45.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/cts.ko 2023-03-03T06:09:45.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/des_generic.ko 2023-03-03T06:09:45.488 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/dh_generic.ko 2023-03-03T06:09:45.489 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/echainiv.ko 2023-03-03T06:09:45.489 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/essiv.ko 2023-03-03T06:09:45.489 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/fcrypt.ko 2023-03-03T06:09:45.489 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/khazad.ko 2023-03-03T06:09:45.489 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/lrw.ko 2023-03-03T06:09:45.489 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/lz4.ko 2023-03-03T06:09:45.489 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/lz4hc.ko 2023-03-03T06:09:45.490 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/md4.ko 2023-03-03T06:09:45.490 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/michael_mic.ko 2023-03-03T06:09:45.490 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/pcbc.ko 2023-03-03T06:09:45.490 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/pcrypt.ko 2023-03-03T06:09:45.490 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/rmd160.ko 2023-03-03T06:09:45.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/seed.ko 2023-03-03T06:09:45.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/serpent_generic.ko 2023-03-03T06:09:45.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 145489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/tcrypt.ko 2023-03-03T06:09:45.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/tea.ko 2023-03-03T06:09:45.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/twofish_common.ko 2023-03-03T06:09:45.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/twofish_generic.ko 2023-03-03T06:09:45.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/vmac.ko 2023-03-03T06:09:45.491 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/wp512.ko 2023-03-03T06:09:45.492 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/xcbc.ko 2023-03-03T06:09:45.492 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/xor.ko 2023-03-03T06:09:45.492 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/xts.ko 2023-03-03T06:09:45.492 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/xxhash_generic.ko 2023-03-03T06:09:45.492 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ 2023-03-03T06:09:45.492 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/ 2023-03-03T06:09:45.492 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/acpi_extlog.ko 2023-03-03T06:09:45.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/acpi_ipmi.ko 2023-03-03T06:09:45.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/acpi_pad.ko 2023-03-03T06:09:45.493 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/apei/ 2023-03-03T06:09:45.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/apei/einj.ko 2023-03-03T06:09:45.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/ec_sys.ko 2023-03-03T06:09:45.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/platform_profile.ko 2023-03-03T06:09:45.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/sbs.ko 2023-03-03T06:09:45.493 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/sbshc.ko 2023-03-03T06:09:45.494 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 99985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/video.ko 2023-03-03T06:09:45.494 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/ 2023-03-03T06:09:45.494 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/acard-ahci.ko 2023-03-03T06:09:45.494 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 89905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/ahci.ko 2023-03-03T06:09:45.494 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/ahci_platform.ko 2023-03-03T06:09:45.494 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 86265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/libahci.ko 2023-03-03T06:09:45.494 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/libahci_platform.ko 2023-03-03T06:09:45.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_acpi.ko 2023-03-03T06:09:45.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_ali.ko 2023-03-03T06:09:45.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_amd.ko 2023-03-03T06:09:45.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_artop.ko 2023-03-03T06:09:45.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_atiixp.ko 2023-03-03T06:09:45.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_atp867x.ko 2023-03-03T06:09:45.495 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_cmd640.ko 2023-03-03T06:09:45.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_cmd64x.ko 2023-03-03T06:09:45.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_cypress.ko 2023-03-03T06:09:45.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_efar.ko 2023-03-03T06:09:45.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_hpt366.ko 2023-03-03T06:09:45.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_hpt37x.ko 2023-03-03T06:09:45.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23073 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-03-03T06:09:45.496 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_hpt3x3.ko 2023-03-03T06:09:45.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_it8213.ko 2023-03-03T06:09:45.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_it821x.ko 2023-03-03T06:09:45.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_jmicron.ko 2023-03-03T06:09:45.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_legacy.ko 2023-03-03T06:09:45.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_marvell.ko 2023-03-03T06:09:45.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_mpiix.ko 2023-03-03T06:09:45.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_netcell.ko 2023-03-03T06:09:45.497 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_ninja32.ko 2023-03-03T06:09:45.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_ns87410.ko 2023-03-03T06:09:45.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_ns87415.ko 2023-03-03T06:09:45.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_oldpiix.ko 2023-03-03T06:09:45.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_opti.ko 2023-03-03T06:09:45.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_optidma.ko 2023-03-03T06:09:45.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_pcmcia.ko 2023-03-03T06:09:45.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_pdc2027x.ko 2023-03-03T06:09:45.498 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-03-03T06:09:45.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_piccolo.ko 2023-03-03T06:09:45.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_radisys.ko 2023-03-03T06:09:45.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_rdc.ko 2023-03-03T06:09:45.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_rz1000.ko 2023-03-03T06:09:45.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_sch.ko 2023-03-03T06:09:45.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_serverworks.ko 2023-03-03T06:09:45.499 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_sil680.ko 2023-03-03T06:09:45.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_sl82c105.ko 2023-03-03T06:09:45.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_triflex.ko 2023-03-03T06:09:45.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_via.ko 2023-03-03T06:09:45.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pdc_adma.ko 2023-03-03T06:09:45.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_inic162x.ko 2023-03-03T06:09:45.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 84481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_mv.ko 2023-03-03T06:09:45.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 72313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_nv.ko 2023-03-03T06:09:45.500 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_promise.ko 2023-03-03T06:09:45.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_qstor.ko 2023-03-03T06:09:45.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_sil.ko 2023-03-03T06:09:45.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_sil24.ko 2023-03-03T06:09:45.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_sis.ko 2023-03-03T06:09:45.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_svw.ko 2023-03-03T06:09:45.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_sx4.ko 2023-03-03T06:09:45.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_uli.ko 2023-03-03T06:09:45.501 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_via.ko 2023-03-03T06:09:45.502 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_vsc.ko 2023-03-03T06:09:45.502 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/auxdisplay/ 2023-03-03T06:09:45.502 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/auxdisplay/ks0108.ko 2023-03-03T06:09:45.502 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/bcma/ 2023-03-03T06:09:45.502 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 123145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/bcma/bcma.ko 2023-03-03T06:09:45.502 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/ 2023-03-03T06:09:45.502 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/aoe/ 2023-03-03T06:09:45.503 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 115577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/aoe/aoe.ko 2023-03-03T06:09:45.503 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/drbd/ 2023-03-03T06:09:45.503 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 980033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/drbd/drbd.ko 2023-03-03T06:09:45.537 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 175665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/floppy.ko 2023-03-03T06:09:45.537 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/mtip32xx/ 2023-03-03T06:09:45.538 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 97825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-03-03T06:09:45.538 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 120025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/nbd.ko 2023-03-03T06:09:45.538 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/null_blk/ 2023-03-03T06:09:45.538 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 111145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/null_blk/null_blk.ko 2023-03-03T06:09:45.538 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 84001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/pktcdvd.ko 2023-03-03T06:09:45.538 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 262345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/rbd.ko 2023-03-03T06:09:45.538 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/xen-blkback/ 2023-03-03T06:09:45.539 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 100049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-03-03T06:09:45.539 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/zram/ 2023-03-03T06:09:45.539 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 64345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/zram/zram.ko 2023-03-03T06:09:45.539 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ 2023-03-03T06:09:45.539 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/applicom.ko 2023-03-03T06:09:45.539 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hangcheck-timer.ko 2023-03-03T06:09:45.539 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hw_random/ 2023-03-03T06:09:45.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hw_random/amd-rng.ko 2023-03-03T06:09:45.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hw_random/intel-rng.ko 2023-03-03T06:09:45.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-03-03T06:09:45.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hw_random/via-rng.ko 2023-03-03T06:09:45.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-03-03T06:09:45.540 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ 2023-03-03T06:09:45.540 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-03-03T06:09:45.541 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 150129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-03-03T06:09:45.541 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-03-03T06:09:45.541 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 142969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-03-03T06:09:45.541 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 71737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-03-03T06:09:45.541 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-03-03T06:09:45.541 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/lp.ko 2023-03-03T06:09:45.541 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/mwave/ 2023-03-03T06:09:45.541 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 91993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/mwave/mwave.ko 2023-03-03T06:09:45.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/nvram.ko 2023-03-03T06:09:45.542 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/pcmcia/ 2023-03-03T06:09:45.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 102753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/pcmcia/cm4000_cs.ko 2023-03-03T06:09:45.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/pcmcia/cm4040_cs.ko 2023-03-03T06:09:45.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/pcmcia/synclink_cs.ko 2023-03-03T06:09:45.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ppdev.ko 2023-03-03T06:09:45.542 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tlclk.ko 2023-03-03T06:09:45.543 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/ 2023-03-03T06:09:45.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-03-03T06:09:45.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-03-03T06:09:45.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-03-03T06:09:45.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-03-03T06:09:45.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-03-03T06:09:45.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-03-03T06:09:45.543 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-03-03T06:09:45.544 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/xillybus/ 2023-03-03T06:09:45.544 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/xillybus/xillybus_class.ko 2023-03-03T06:09:45.544 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/xillybus/xillybus_core.ko 2023-03-03T06:09:45.544 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-03-03T06:09:45.544 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/clk/ 2023-03-03T06:09:45.544 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/clk/clk-palmas.ko 2023-03-03T06:09:45.544 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/clk/clk-si5351.ko 2023-03-03T06:09:45.545 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/clk/clk-twl6040.ko 2023-03-03T06:09:45.545 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/clk/clk-wm831x.ko 2023-03-03T06:09:45.545 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/cpufreq/ 2023-03-03T06:09:45.545 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-03-03T06:09:45.545 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-03-03T06:09:45.545 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-03-03T06:09:45.545 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/ 2023-03-03T06:09:45.545 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/ccp/ 2023-03-03T06:09:45.546 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-03-03T06:09:45.546 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 225865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/ccp/ccp.ko 2023-03-03T06:09:45.546 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/padlock-aes.ko 2023-03-03T06:09:45.546 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/padlock-sha.ko 2023-03-03T06:09:45.546 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/qat/ 2023-03-03T06:09:45.546 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/qat/qat_common/ 2023-03-03T06:09:45.546 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 415785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/qat/qat_common/intel_qat.ko 2023-03-03T06:09:45.547 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/qat/qat_dh895xcc/ 2023-03-03T06:09:45.547 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-03-03T06:09:45.547 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/virtio/ 2023-03-03T06:09:45.547 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 71609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-03-03T06:09:45.547 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dca/ 2023-03-03T06:09:45.547 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dca/dca.ko 2023-03-03T06:09:45.547 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dma/ 2023-03-03T06:09:45.547 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dma/dw/ 2023-03-03T06:09:45.548 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dma/dw/dw_dmac.ko 2023-03-03T06:09:45.548 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dma/ioat/ 2023-03-03T06:09:45.548 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 124625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dma/ioat/ioatdma.ko 2023-03-03T06:09:45.548 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/ 2023-03-03T06:09:45.548 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 74129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/amd64_edac.ko 2023-03-03T06:09:45.548 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/e752x_edac.ko 2023-03-03T06:09:45.548 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 65753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/edac_mce_amd.ko 2023-03-03T06:09:45.549 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i3000_edac.ko 2023-03-03T06:09:45.549 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i3200_edac.ko 2023-03-03T06:09:45.549 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i5100_edac.ko 2023-03-03T06:09:45.549 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i5400_edac.ko 2023-03-03T06:09:45.549 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i7300_edac.ko 2023-03-03T06:09:45.549 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i7core_edac.ko 2023-03-03T06:09:45.549 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i82975x_edac.ko 2023-03-03T06:09:45.549 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/ie31200_edac.ko 2023-03-03T06:09:45.550 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/sb_edac.ko 2023-03-03T06:09:45.550 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/x38_edac.ko 2023-03-03T06:09:45.550 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/ 2023-03-03T06:09:45.550 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-gpio.ko 2023-03-03T06:09:45.550 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-max14577.ko 2023-03-03T06:09:45.550 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-max77693.ko 2023-03-03T06:09:45.550 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-max8997.ko 2023-03-03T06:09:45.551 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-palmas.ko 2023-03-03T06:09:45.551 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-rt8973a.ko 2023-03-03T06:09:45.551 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-sm5502.ko 2023-03-03T06:09:45.551 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/firmware/ 2023-03-03T06:09:45.551 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/firmware/dmi-sysfs.ko 2023-03-03T06:09:45.551 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/firmware/efi/ 2023-03-03T06:09:45.551 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/firmware/efi/efi-pstore.ko 2023-03-03T06:09:45.551 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/firmware/iscsi_ibft.ko 2023-03-03T06:09:45.552 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/ 2023-03-03T06:09:45.552 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-adp5520.ko 2023-03-03T06:09:45.552 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-amd8111.ko 2023-03-03T06:09:45.552 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-arizona.ko 2023-03-03T06:09:45.552 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-da9052.ko 2023-03-03T06:09:45.552 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-da9055.ko 2023-03-03T06:09:45.552 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-dln2.ko 2023-03-03T06:09:45.552 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 78745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-f7188x.ko 2023-03-03T06:09:45.553 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-generic.ko 2023-03-03T06:09:45.553 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-ich.ko 2023-03-03T06:09:45.553 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-03-03T06:09:45.553 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-kempld.ko 2023-03-03T06:09:45.553 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-lp3943.ko 2023-03-03T06:09:45.553 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-max7300.ko 2023-03-03T06:09:45.553 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-max730x.ko 2023-03-03T06:09:45.554 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-max732x.ko 2023-03-03T06:09:45.554 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-03-03T06:09:45.554 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-pca953x.ko 2023-03-03T06:09:45.554 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-pcf857x.ko 2023-03-03T06:09:45.554 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-rdc321x.ko 2023-03-03T06:09:45.554 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-sch.ko 2023-03-03T06:09:45.554 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-sch311x.ko 2023-03-03T06:09:45.555 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-tps65912.ko 2023-03-03T06:09:45.555 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-twl4030.ko 2023-03-03T06:09:45.555 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-twl6040.ko 2023-03-03T06:09:45.555 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-viperboard.ko 2023-03-03T06:09:45.555 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-vx855.ko 2023-03-03T06:09:45.555 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-wm831x.ko 2023-03-03T06:09:45.555 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-wm8350.ko 2023-03-03T06:09:45.556 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-wm8994.ko 2023-03-03T06:09:45.556 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/ 2023-03-03T06:09:45.556 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-a4tech.ko 2023-03-03T06:09:45.556 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-apple.ko 2023-03-03T06:09:45.556 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-appleir.ko 2023-03-03T06:09:45.556 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-aureal.ko 2023-03-03T06:09:45.556 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-axff.ko 2023-03-03T06:09:45.556 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-belkin.ko 2023-03-03T06:09:45.557 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-cherry.ko 2023-03-03T06:09:45.557 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-chicony.ko 2023-03-03T06:09:45.557 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-cp2112.ko 2023-03-03T06:09:45.557 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-cypress.ko 2023-03-03T06:09:45.557 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-dr.ko 2023-03-03T06:09:45.557 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-elecom.ko 2023-03-03T06:09:45.557 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-elo.ko 2023-03-03T06:09:45.558 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-emsff.ko 2023-03-03T06:09:45.558 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-ezkey.ko 2023-03-03T06:09:45.558 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-gaff.ko 2023-03-03T06:09:45.558 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-generic.ko 2023-03-03T06:09:45.558 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-gt683r.ko 2023-03-03T06:09:45.558 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-gyration.ko 2023-03-03T06:09:45.558 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-03-03T06:09:45.558 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-03-03T06:09:45.559 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-holtekff.ko 2023-03-03T06:09:45.559 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-hyperv.ko 2023-03-03T06:09:45.559 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-icade.ko 2023-03-03T06:09:45.559 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-kensington.ko 2023-03-03T06:09:45.559 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-keytouch.ko 2023-03-03T06:09:45.559 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-kye.ko 2023-03-03T06:09:45.559 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-lcpower.ko 2023-03-03T06:09:45.559 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-led.ko 2023-03-03T06:09:45.560 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-lenovo.ko 2023-03-03T06:09:45.560 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-lg-g15.ko 2023-03-03T06:09:45.560 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-logitech-dj.ko 2023-03-03T06:09:45.560 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 89577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-03-03T06:09:45.560 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 88385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-logitech.ko 2023-03-03T06:09:45.560 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-magicmouse.ko 2023-03-03T06:09:45.560 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-microsoft.ko 2023-03-03T06:09:45.561 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-monterey.ko 2023-03-03T06:09:45.561 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-multitouch.ko 2023-03-03T06:09:45.561 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-ntrig.ko 2023-03-03T06:09:45.561 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-ortek.ko 2023-03-03T06:09:45.561 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-penmount.ko 2023-03-03T06:09:45.561 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-petalynx.ko 2023-03-03T06:09:45.561 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 71273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-picolcd.ko 2023-03-03T06:09:45.562 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-pl.ko 2023-03-03T06:09:45.562 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-plantronics.ko 2023-03-03T06:09:45.562 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-primax.ko 2023-03-03T06:09:45.562 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-rmi.ko 2023-03-03T06:09:45.562 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-03-03T06:09:45.562 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-common.ko 2023-03-03T06:09:45.562 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-isku.ko 2023-03-03T06:09:45.562 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-kone.ko 2023-03-03T06:09:45.563 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-03-03T06:09:45.563 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-03-03T06:09:45.563 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-03-03T06:09:45.563 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-lua.ko 2023-03-03T06:09:45.563 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-03-03T06:09:45.563 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-03-03T06:09:45.563 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-savu.ko 2023-03-03T06:09:45.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat.ko 2023-03-03T06:09:45.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-saitek.ko 2023-03-03T06:09:45.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-samsung.ko 2023-03-03T06:09:45.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-sensor-hub.ko 2023-03-03T06:09:45.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-sjoy.ko 2023-03-03T06:09:45.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 72945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-sony.ko 2023-03-03T06:09:45.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-speedlink.ko 2023-03-03T06:09:45.564 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-steelseries.ko 2023-03-03T06:09:45.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-sunplus.ko 2023-03-03T06:09:45.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-thrustmaster.ko 2023-03-03T06:09:45.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-tivo.ko 2023-03-03T06:09:45.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-tmff.ko 2023-03-03T06:09:45.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-topseed.ko 2023-03-03T06:09:45.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-twinhan.ko 2023-03-03T06:09:45.565 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 68433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-uclogic.ko 2023-03-03T06:09:45.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-waltop.ko 2023-03-03T06:09:45.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 145113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-wiimote.ko 2023-03-03T06:09:45.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-xinmo.ko 2023-03-03T06:09:45.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-zpff.ko 2023-03-03T06:09:45.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-zydacron.ko 2023-03-03T06:09:45.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 260569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid.ko 2023-03-03T06:09:45.566 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/uhid.ko 2023-03-03T06:09:45.566 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/usbhid/ 2023-03-03T06:09:45.567 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 126433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/usbhid/usbhid.ko 2023-03-03T06:09:45.567 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/usbhid/usbkbd.ko 2023-03-03T06:09:45.567 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/usbhid/usbmouse.ko 2023-03-03T06:09:45.567 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 218593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/wacom.ko 2023-03-03T06:09:45.567 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hv/ 2023-03-03T06:09:45.567 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 65753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hv/hv_balloon.ko 2023-03-03T06:09:45.567 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 92873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hv/hv_utils.ko 2023-03-03T06:09:45.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 309889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hv/hv_vmbus.ko 2023-03-03T06:09:45.568 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ 2023-03-03T06:09:45.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/abituguru.ko 2023-03-03T06:09:45.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/abituguru3.ko 2023-03-03T06:09:45.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-03-03T06:09:45.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ad7414.ko 2023-03-03T06:09:45.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ad7418.ko 2023-03-03T06:09:45.568 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adc128d818.ko 2023-03-03T06:09:45.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adm1025.ko 2023-03-03T06:09:45.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 98161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adm1026.ko 2023-03-03T06:09:45.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adm1029.ko 2023-03-03T06:09:45.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adm1031.ko 2023-03-03T06:09:45.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adm9240.ko 2023-03-03T06:09:45.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ads7828.ko 2023-03-03T06:09:45.569 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adt7410.ko 2023-03-03T06:09:45.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adt7411.ko 2023-03-03T06:09:45.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 74009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adt7462.ko 2023-03-03T06:09:45.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adt7470.ko 2023-03-03T06:09:45.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 65281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adt7475.ko 2023-03-03T06:09:45.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adt7x10.ko 2023-03-03T06:09:45.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/amc6821.ko 2023-03-03T06:09:45.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/applesmc.ko 2023-03-03T06:09:45.570 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/asb100.ko 2023-03-03T06:09:45.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/asc7621.ko 2023-03-03T06:09:45.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/asus_atk0110.ko 2023-03-03T06:09:45.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/atxp1.ko 2023-03-03T06:09:45.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/coretemp.ko 2023-03-03T06:09:45.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/da9052-hwmon.ko 2023-03-03T06:09:45.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/da9055-hwmon.ko 2023-03-03T06:09:45.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-03-03T06:09:45.571 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 106217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/dme1737.ko 2023-03-03T06:09:45.572 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ds1621.ko 2023-03-03T06:09:45.572 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ds620.ko 2023-03-03T06:09:45.572 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/emc1403.ko 2023-03-03T06:09:45.572 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/emc2103.ko 2023-03-03T06:09:45.572 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/emc6w201.ko 2023-03-03T06:09:45.572 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/f71805f.ko 2023-03-03T06:09:45.572 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 94609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/f71882fg.ko 2023-03-03T06:09:45.572 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/f75375s.ko 2023-03-03T06:09:45.573 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/fam15h_power.ko 2023-03-03T06:09:45.573 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/fschmd.ko 2023-03-03T06:09:45.573 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/g760a.ko 2023-03-03T06:09:45.573 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/g762.ko 2023-03-03T06:09:45.573 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/gl518sm.ko 2023-03-03T06:09:45.573 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/gl520sm.ko 2023-03-03T06:09:45.573 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/hih6130.ko 2023-03-03T06:09:45.574 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/hwmon-vid.ko 2023-03-03T06:09:45.574 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/i5500_temp.ko 2023-03-03T06:09:45.574 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/i5k_amb.ko 2023-03-03T06:09:45.574 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ibmaem.ko 2023-03-03T06:09:45.574 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ibmpex.ko 2023-03-03T06:09:45.574 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ina209.ko 2023-03-03T06:09:45.574 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ina2xx.ko 2023-03-03T06:09:45.575 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 118873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/it87.ko 2023-03-03T06:09:45.575 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/jc42.ko 2023-03-03T06:09:45.575 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/k10temp.ko 2023-03-03T06:09:45.575 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/k8temp.ko 2023-03-03T06:09:45.575 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lineage-pem.ko 2023-03-03T06:09:45.575 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm63.ko 2023-03-03T06:09:45.575 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm73.ko 2023-03-03T06:09:45.575 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm75.ko 2023-03-03T06:09:45.576 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm77.ko 2023-03-03T06:09:45.576 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm78.ko 2023-03-03T06:09:45.576 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm80.ko 2023-03-03T06:09:45.576 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm83.ko 2023-03-03T06:09:45.576 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 72785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm85.ko 2023-03-03T06:09:45.576 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm87.ko 2023-03-03T06:09:45.576 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm90.ko 2023-03-03T06:09:45.577 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm92.ko 2023-03-03T06:09:45.577 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 112985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm93.ko 2023-03-03T06:09:45.577 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm95234.ko 2023-03-03T06:09:45.577 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm95241.ko 2023-03-03T06:09:45.577 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm95245.ko 2023-03-03T06:09:45.577 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc2945.ko 2023-03-03T06:09:45.577 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc4151.ko 2023-03-03T06:09:45.578 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc4215.ko 2023-03-03T06:09:45.578 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc4222.ko 2023-03-03T06:09:45.578 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc4245.ko 2023-03-03T06:09:45.579 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc4260.ko 2023-03-03T06:09:45.579 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc4261.ko 2023-03-03T06:09:45.579 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max16065.ko 2023-03-03T06:09:45.579 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max1619.ko 2023-03-03T06:09:45.579 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max1668.ko 2023-03-03T06:09:45.579 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max197.ko 2023-03-03T06:09:45.579 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max6639.ko 2023-03-03T06:09:45.579 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max6650.ko 2023-03-03T06:09:45.580 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max6697.ko 2023-03-03T06:09:45.580 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/mc13783-adc.ko 2023-03-03T06:09:45.580 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/mcp3021.ko 2023-03-03T06:09:45.580 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-03-03T06:09:45.580 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/nct6683.ko 2023-03-03T06:09:45.580 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 114801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/nct6775-core.ko 2023-03-03T06:09:45.580 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/nct6775.ko 2023-03-03T06:09:45.580 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/nct7802.ko 2023-03-03T06:09:45.581 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 81465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pc87360.ko 2023-03-03T06:09:45.581 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pc87427.ko 2023-03-03T06:09:45.581 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pcf8591.ko 2023-03-03T06:09:45.581 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/ 2023-03-03T06:09:45.581 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-03-03T06:09:45.581 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-03-03T06:09:45.581 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-03-03T06:09:45.582 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-03-03T06:09:45.582 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-03-03T06:09:45.582 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-03-03T06:09:45.582 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-03-03T06:09:45.582 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 72265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-03-03T06:09:45.582 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-03-03T06:09:45.582 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-03-03T06:09:45.582 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-03-03T06:09:45.583 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-03-03T06:09:45.583 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/powr1220.ko 2023-03-03T06:09:45.583 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/sch5627.ko 2023-03-03T06:09:45.583 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/sch5636.ko 2023-03-03T06:09:45.583 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/sch56xx-common.ko 2023-03-03T06:09:45.583 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/sht15.ko 2023-03-03T06:09:45.583 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/sht21.ko 2023-03-03T06:09:45.584 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/shtc1.ko 2023-03-03T06:09:45.584 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/sis5595.ko 2023-03-03T06:09:45.584 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/smm665.ko 2023-03-03T06:09:45.584 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/smsc47b397.ko 2023-03-03T06:09:45.584 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/smsc47m1.ko 2023-03-03T06:09:45.584 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/smsc47m192.ko 2023-03-03T06:09:45.584 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/thmc50.ko 2023-03-03T06:09:45.585 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/tmp102.ko 2023-03-03T06:09:45.585 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/tmp103.ko 2023-03-03T06:09:45.585 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/tmp401.ko 2023-03-03T06:09:45.585 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/tmp421.ko 2023-03-03T06:09:45.585 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/via-cputemp.ko 2023-03-03T06:09:45.585 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/via686a.ko 2023-03-03T06:09:45.585 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/vt1211.ko 2023-03-03T06:09:45.586 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/vt8231.ko 2023-03-03T06:09:45.586 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 57337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83627ehf.ko 2023-03-03T06:09:45.586 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 79145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83627hf.ko 2023-03-03T06:09:45.586 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 72073 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83781d.ko 2023-03-03T06:09:45.586 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83791d.ko 2023-03-03T06:09:45.586 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 71273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83792d.ko 2023-03-03T06:09:45.586 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 101769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83793.ko 2023-03-03T06:09:45.586 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 119897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83795.ko 2023-03-03T06:09:45.587 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83l785ts.ko 2023-03-03T06:09:45.587 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83l786ng.ko 2023-03-03T06:09:45.587 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-03-03T06:09:45.587 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-03-03T06:09:45.587 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/ 2023-03-03T06:09:45.587 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/algos/ 2023-03-03T06:09:45.587 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-03-03T06:09:45.588 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-03-03T06:09:45.588 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/ 2023-03-03T06:09:45.588 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-03-03T06:09:45.588 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-03-03T06:09:45.588 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-03-03T06:09:45.588 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-03-03T06:09:45.588 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-03-03T06:09:45.588 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-03-03T06:09:45.589 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-03-03T06:09:45.589 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-03-03T06:09:45.589 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-03-03T06:09:45.589 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-03-03T06:09:45.589 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-03-03T06:09:45.589 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-03-03T06:09:45.589 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-03-03T06:09:45.589 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-03-03T06:09:45.590 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-03-03T06:09:45.590 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-03-03T06:09:45.590 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-03-03T06:09:45.590 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-03-03T06:09:45.590 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-03-03T06:09:45.590 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-03-03T06:09:45.590 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-03-03T06:09:45.591 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-03-03T06:09:45.591 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-03-03T06:09:45.591 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-03-03T06:09:45.591 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-03-03T06:09:45.591 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-03-03T06:09:45.591 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-03-03T06:09:45.591 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-03-03T06:09:45.591 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-03-03T06:09:45.592 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-03-03T06:09:45.592 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-03-03T06:09:45.592 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-03-03T06:09:45.592 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-03-03T06:09:45.592 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-via.ko 2023-03-03T06:09:45.592 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-03-03T06:09:45.592 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-03-03T06:09:45.593 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-03-03T06:09:45.593 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/i2c-mux.ko 2023-03-03T06:09:45.593 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/i2c-smbus.ko 2023-03-03T06:09:45.593 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/i2c-stub.ko 2023-03-03T06:09:45.593 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/muxes/ 2023-03-03T06:09:45.593 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-03-03T06:09:45.593 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-03-03T06:09:45.594 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-03-03T06:09:45.594 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ 2023-03-03T06:09:45.594 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/ 2023-03-03T06:09:45.594 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 274945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/ib_cm.ko 2023-03-03T06:09:45.594 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 863201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/ib_core.ko 2023-03-03T06:09:45.594 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/ib_umad.ko 2023-03-03T06:09:45.594 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 321169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-03-03T06:09:45.594 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 100529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/iw_cm.ko 2023-03-03T06:09:45.595 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 276617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-03-03T06:09:45.595 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 65433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-03-03T06:09:45.595 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/ 2023-03-03T06:09:45.595 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/cxgb4/ 2023-03-03T06:09:45.595 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 499489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-03-03T06:09:45.595 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/mlx4/ 2023-03-03T06:09:45.595 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 425945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-03-03T06:09:45.595 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/mlx5/ 2023-03-03T06:09:45.596 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 734337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-03-03T06:09:45.596 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/mthca/ 2023-03-03T06:09:45.596 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 302809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-03-03T06:09:45.596 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/ocrdma/ 2023-03-03T06:09:45.596 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 157289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-03-03T06:09:45.596 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/usnic/ 2023-03-03T06:09:45.596 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 178337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-03-03T06:09:45.596 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/ 2023-03-03T06:09:45.597 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/ipoib/ 2023-03-03T06:09:45.597 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 272561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-03-03T06:09:45.597 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/iser/ 2023-03-03T06:09:45.597 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 106273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-03-03T06:09:45.597 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/isert/ 2023-03-03T06:09:45.597 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 119601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-03-03T06:09:45.597 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/srp/ 2023-03-03T06:09:45.598 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 134625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-03-03T06:09:45.598 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/srpt/ 2023-03-03T06:09:45.598 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 134249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-03-03T06:09:45.598 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/ 2023-03-03T06:09:45.598 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/evbug.ko 2023-03-03T06:09:45.598 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/ff-memless.ko 2023-03-03T06:09:45.598 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/gameport/ 2023-03-03T06:09:45.598 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-03-03T06:09:45.599 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/gameport/fm801-gp.ko 2023-03-03T06:09:45.599 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/gameport/gameport.ko 2023-03-03T06:09:45.599 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/gameport/lightning.ko 2023-03-03T06:09:45.599 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/gameport/ns558.ko 2023-03-03T06:09:45.599 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/joydev.ko 2023-03-03T06:09:45.599 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/ 2023-03-03T06:09:45.599 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-03-03T06:09:45.600 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-03-03T06:09:45.600 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-03-03T06:09:45.600 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-03-03T06:09:45.600 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-03-03T06:09:45.600 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-03-03T06:09:45.600 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/lkkbd.ko 2023-03-03T06:09:45.600 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/lm8323.ko 2023-03-03T06:09:45.600 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/lm8333.ko 2023-03-03T06:09:45.601 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-03-03T06:09:45.601 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-03-03T06:09:45.601 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-03-03T06:09:45.601 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-03-03T06:09:45.601 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/newtonkbd.ko 2023-03-03T06:09:45.601 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-03-03T06:09:45.601 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/qt1070.ko 2023-03-03T06:09:45.602 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/qt2160.ko 2023-03-03T06:09:45.602 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-03-03T06:09:45.602 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/stowaway.ko 2023-03-03T06:09:45.602 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/sunkbd.ko 2023-03-03T06:09:45.602 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-03-03T06:09:45.602 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-03-03T06:09:45.602 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-03-03T06:09:45.602 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/xtkbd.ko 2023-03-03T06:09:45.603 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/matrix-keymap.ko 2023-03-03T06:09:45.603 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/ 2023-03-03T06:09:45.603 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/appletouch.ko 2023-03-03T06:09:45.603 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/bcm5974.ko 2023-03-03T06:09:45.603 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 126353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/cyapatp.ko 2023-03-03T06:09:45.603 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 95897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/elan_i2c.ko 2023-03-03T06:09:45.603 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/gpio_mouse.ko 2023-03-03T06:09:45.603 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 346145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/psmouse.ko 2023-03-03T06:09:45.604 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/sermouse.ko 2023-03-03T06:09:45.604 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-03-03T06:09:45.604 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-03-03T06:09:45.604 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-03-03T06:09:45.604 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/rmi4/ 2023-03-03T06:09:45.604 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 113481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/rmi4/rmi_core.ko 2023-03-03T06:09:45.604 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/ 2023-03-03T06:09:45.605 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/altera_ps2.ko 2023-03-03T06:09:45.605 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/arc_ps2.ko 2023-03-03T06:09:45.605 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/ct82c710.ko 2023-03-03T06:09:45.605 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-03-03T06:09:45.605 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/parkbd.ko 2023-03-03T06:09:45.605 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/pcips2.ko 2023-03-03T06:09:45.605 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/ps2mult.ko 2023-03-03T06:09:45.605 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/serio_raw.ko 2023-03-03T06:09:45.606 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/serport.ko 2023-03-03T06:09:45.606 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/sparse-keymap.ko 2023-03-03T06:09:45.606 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/iommu/ 2023-03-03T06:09:45.606 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/iommu/amd/ 2023-03-03T06:09:45.606 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-03-03T06:09:45.606 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/ 2023-03-03T06:09:45.606 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-88pm860x.ko 2023-03-03T06:09:45.607 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-adp5520.ko 2023-03-03T06:09:45.607 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-bd2802.ko 2023-03-03T06:09:45.607 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-blinkm.ko 2023-03-03T06:09:45.607 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-da903x.ko 2023-03-03T06:09:45.607 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-da9052.ko 2023-03-03T06:09:45.607 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-gpio.ko 2023-03-03T06:09:45.607 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-lm3530.ko 2023-03-03T06:09:45.607 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-lm3533.ko 2023-03-03T06:09:45.608 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-lm355x.ko 2023-03-03T06:09:45.608 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-lm3642.ko 2023-03-03T06:09:45.608 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-lp3944.ko 2023-03-03T06:09:45.608 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-lp8788.ko 2023-03-03T06:09:45.608 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-max8997.ko 2023-03-03T06:09:45.608 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-mc13783.ko 2023-03-03T06:09:45.608 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-menf21bmc.ko 2023-03-03T06:09:45.608 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-pca9532.ko 2023-03-03T06:09:45.609 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-pca955x.ko 2023-03-03T06:09:45.609 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-pca963x.ko 2023-03-03T06:09:45.609 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-ss4200.ko 2023-03-03T06:09:45.609 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-tca6507.ko 2023-03-03T06:09:45.609 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-wm831x-status.ko 2023-03-03T06:09:45.609 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-wm8350.ko 2023-03-03T06:09:45.609 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ 2023-03-03T06:09:45.610 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-03-03T06:09:45.610 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-03-03T06:09:45.610 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-03-03T06:09:45.610 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-03-03T06:09:45.610 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2023-03-03T06:09:45.610 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-03-03T06:09:45.610 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-03-03T06:09:45.610 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-03-03T06:09:45.611 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-03-03T06:09:45.611 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/ 2023-03-03T06:09:45.611 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/bcache/ 2023-03-03T06:09:45.611 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 650041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/bcache/bcache.ko 2023-03-03T06:09:45.611 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-bio-prison.ko 2023-03-03T06:09:45.611 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 80177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-bufio.ko 2023-03-03T06:09:45.611 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-cache-smq.ko 2023-03-03T06:09:45.612 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 156121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-cache.ko 2023-03-03T06:09:45.612 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 109385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-crypt.ko 2023-03-03T06:09:45.612 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-delay.ko 2023-03-03T06:09:45.612 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-era.ko 2023-03-03T06:09:45.612 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-flakey.ko 2023-03-03T06:09:45.612 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-log-userspace.ko 2023-03-03T06:09:45.612 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-log.ko 2023-03-03T06:09:45.612 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-mirror.ko 2023-03-03T06:09:45.613 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 77673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-multipath.ko 2023-03-03T06:09:45.613 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-queue-length.ko 2023-03-03T06:09:45.613 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 72689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-raid.ko 2023-03-03T06:09:45.613 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-region-hash.ko 2023-03-03T06:09:45.613 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-round-robin.ko 2023-03-03T06:09:45.613 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-service-time.ko 2023-03-03T06:09:45.613 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 111985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-snapshot.ko 2023-03-03T06:09:45.614 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-switch.ko 2023-03-03T06:09:45.614 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 186137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-thin-pool.ko 2023-03-03T06:09:45.614 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-verity.ko 2023-03-03T06:09:45.614 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-zero.ko 2023-03-03T06:09:45.614 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/faulty.ko 2023-03-03T06:09:45.614 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/linear.ko 2023-03-03T06:09:45.614 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/multipath.ko 2023-03-03T06:09:45.615 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/persistent-data/ 2023-03-03T06:09:45.615 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 196865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-03-03T06:09:45.615 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/raid0.ko 2023-03-03T06:09:45.615 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 110553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/raid1.ko 2023-03-03T06:09:45.615 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 156729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/raid10.ko 2023-03-03T06:09:45.615 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 396433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/raid456.ko 2023-03-03T06:09:45.615 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/ 2023-03-03T06:09:45.616 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/ 2023-03-03T06:09:45.616 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 180969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptbase.ko 2023-03-03T06:09:45.616 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptctl.ko 2023-03-03T06:09:45.616 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptfc.ko 2023-03-03T06:09:45.616 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptlan.ko 2023-03-03T06:09:45.616 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 134233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptsas.ko 2023-03-03T06:09:45.616 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 79705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptscsih.ko 2023-03-03T06:09:45.616 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptspi.ko 2023-03-03T06:09:45.617 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/ 2023-03-03T06:09:45.617 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/88pm800.ko 2023-03-03T06:09:45.617 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/88pm805.ko 2023-03-03T06:09:45.617 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/88pm80x.ko 2023-03-03T06:09:45.617 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/arizona-i2c.ko 2023-03-03T06:09:45.617 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 127713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/arizona.ko 2023-03-03T06:09:45.617 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/bcm590xx.ko 2023-03-03T06:09:45.618 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/cros_ec_dev.ko 2023-03-03T06:09:45.618 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/dln2.ko 2023-03-03T06:09:45.618 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/htc-pasic3.ko 2023-03-03T06:09:45.618 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/janz-cmodio.ko 2023-03-03T06:09:45.618 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/kempld-core.ko 2023-03-03T06:09:45.618 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/lm3533-core.ko 2023-03-03T06:09:45.618 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-03-03T06:09:45.619 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/lp3943.ko 2023-03-03T06:09:45.619 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/lpc_ich.ko 2023-03-03T06:09:45.619 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/lpc_sch.ko 2023-03-03T06:09:45.619 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/max8907.ko 2023-03-03T06:09:45.619 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/mc13xxx-core.ko 2023-03-03T06:09:45.619 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-03-03T06:09:45.619 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/menf21bmc.ko 2023-03-03T06:09:45.619 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/pcf50633-adc.ko 2023-03-03T06:09:45.620 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/pcf50633-gpio.ko 2023-03-03T06:09:45.620 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/pcf50633.ko 2023-03-03T06:09:45.620 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-03-03T06:09:45.620 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/retu-mfd.ko 2023-03-03T06:09:45.620 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 77817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/si476x-core.ko 2023-03-03T06:09:45.620 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/sm501.ko 2023-03-03T06:09:45.620 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-03-03T06:09:45.620 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/tps65010.ko 2023-03-03T06:09:45.621 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/tps6507x.ko 2023-03-03T06:09:45.621 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/viperboard.ko 2023-03-03T06:09:45.621 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/vx855.ko 2023-03-03T06:09:45.621 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/wl1273-core.ko 2023-03-03T06:09:45.621 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/wm8994.ko 2023-03-03T06:09:45.621 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ 2023-03-03T06:09:45.621 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-03-03T06:09:45.622 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ad525x_dpot.ko 2023-03-03T06:09:45.622 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/altera-stapl/ 2023-03-03T06:09:45.622 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-03-03T06:09:45.622 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/apds9802als.ko 2023-03-03T06:09:45.622 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/apds990x.ko 2023-03-03T06:09:45.622 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/bh1770glc.ko 2023-03-03T06:09:45.622 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/c2port/ 2023-03-03T06:09:45.622 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-03-03T06:09:45.623 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/c2port/core.ko 2023-03-03T06:09:45.623 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/cb710/ 2023-03-03T06:09:45.623 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/cb710/cb710.ko 2023-03-03T06:09:45.623 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ds1682.ko 2023-03-03T06:09:45.623 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/dummy-irq.ko 2023-03-03T06:09:45.623 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/echo/ 2023-03-03T06:09:45.623 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/echo/echo.ko 2023-03-03T06:09:45.623 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/eeprom/ 2023-03-03T06:09:45.624 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/eeprom/at24.ko 2023-03-03T06:09:45.624 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/eeprom/eeprom.ko 2023-03-03T06:09:45.624 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-03-03T06:09:45.624 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/eeprom/max6875.ko 2023-03-03T06:09:45.624 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/enclosure.ko 2023-03-03T06:09:45.624 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/genwqe/ 2023-03-03T06:09:45.624 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 155889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-03-03T06:09:45.625 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/hmc6352.ko 2023-03-03T06:09:45.625 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/hpilo.ko 2023-03-03T06:09:45.625 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ibmasm/ 2023-03-03T06:09:45.625 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 73553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-03-03T06:09:45.625 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ics932s401.ko 2023-03-03T06:09:45.625 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/isl29003.ko 2023-03-03T06:09:45.625 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/isl29020.ko 2023-03-03T06:09:45.625 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/lis3lv02d/ 2023-03-03T06:09:45.626 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-03-03T06:09:45.626 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-03-03T06:09:45.626 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/mei/ 2023-03-03T06:09:45.626 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/mei/mei-me.ko 2023-03-03T06:09:45.626 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/mei/mei-txe.ko 2023-03-03T06:09:45.626 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 312217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/mei/mei.ko 2023-03-03T06:09:45.626 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/phantom.ko 2023-03-03T06:09:45.626 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ti-st/ 2023-03-03T06:09:45.627 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 75801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ti-st/st_drv.ko 2023-03-03T06:09:45.672 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/tifm_7xx1.ko 2023-03-03T06:09:45.672 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/tifm_core.ko 2023-03-03T06:09:45.673 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/tsl2550.ko 2023-03-03T06:09:45.673 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/vmw_balloon.ko 2023-03-03T06:09:45.673 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/vmw_vmci/ 2023-03-03T06:09:45.673 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 182801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-03-03T06:09:45.673 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ 2023-03-03T06:09:45.673 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/bonding/ 2023-03-03T06:09:45.673 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 488953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/bonding/bonding.ko 2023-03-03T06:09:45.674 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/ 2023-03-03T06:09:45.674 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/b53/ 2023-03-03T06:09:45.674 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 99881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-03-03T06:09:45.674 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-03-03T06:09:45.674 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/mv88e6060.ko 2023-03-03T06:09:45.674 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/mv88e6xxx/ 2023-03-03T06:09:45.674 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 445361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-03-03T06:09:45.675 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dummy.ko 2023-03-03T06:09:45.675 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/eql.ko 2023-03-03T06:09:45.675 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/ 2023-03-03T06:09:45.675 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/3com/ 2023-03-03T06:09:45.675 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-03-03T06:09:45.675 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-03-03T06:09:45.675 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 132169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-03-03T06:09:45.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-03-03T06:09:45.676 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/8390/ 2023-03-03T06:09:45.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/8390/8390.ko 2023-03-03T06:09:45.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-03-03T06:09:45.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-03-03T06:09:45.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 99553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-03-03T06:09:45.676 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/adaptec/ 2023-03-03T06:09:45.676 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-03-03T06:09:45.677 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/agere/ 2023-03-03T06:09:45.677 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-03-03T06:09:45.677 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/alteon/ 2023-03-03T06:09:45.677 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-03-03T06:09:45.677 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/altera/ 2023-03-03T06:09:45.677 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-03-03T06:09:45.677 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/amd/ 2023-03-03T06:09:45.678 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-03-03T06:09:45.678 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-03-03T06:09:45.678 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-03-03T06:09:45.678 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/ 2023-03-03T06:09:45.678 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/alx/ 2023-03-03T06:09:45.678 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 99337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-03-03T06:09:45.678 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-03-03T06:09:45.678 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 106705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-03-03T06:09:45.679 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-03-03T06:09:45.679 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 91945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-03-03T06:09:45.679 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atlx/ 2023-03-03T06:09:45.679 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 82753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-03-03T06:09:45.679 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-03-03T06:09:45.679 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/ 2023-03-03T06:09:45.679 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 79497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-03-03T06:09:45.679 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 176809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-03-03T06:09:45.680 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-03-03T06:09:45.680 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1535233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-03-03T06:09:45.680 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 130009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-03-03T06:09:45.680 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/genet/ 2023-03-03T06:09:45.680 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 133305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-03-03T06:09:45.680 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 316225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-03-03T06:09:45.680 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/brocade/ 2023-03-03T06:09:45.680 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/brocade/bna/ 2023-03-03T06:09:45.681 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 334513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-03-03T06:09:45.681 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/ 2023-03-03T06:09:45.681 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-03-03T06:09:45.681 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 194849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-03-03T06:09:45.681 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-03-03T06:09:45.681 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 353761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-03-03T06:09:45.681 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-03-03T06:09:45.682 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1061553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-03-03T06:09:45.682 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-03-03T06:09:45.682 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 169497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-03-03T06:09:45.682 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-03-03T06:09:45.682 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-03-03T06:09:45.682 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/cisco/ 2023-03-03T06:09:45.682 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/cisco/enic/ 2023-03-03T06:09:45.683 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 209577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-03-03T06:09:45.683 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/ 2023-03-03T06:09:45.683 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/ 2023-03-03T06:09:45.683 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-03-03T06:09:45.683 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-03-03T06:09:45.683 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 145593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-03-03T06:09:45.683 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-03-03T06:09:45.683 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 64153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-03-03T06:09:45.684 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-03-03T06:09:45.684 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dlink/ 2023-03-03T06:09:45.684 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-03-03T06:09:45.684 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 64129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-03-03T06:09:45.684 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dnet.ko 2023-03-03T06:09:45.684 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/ec_bhf.ko 2023-03-03T06:09:45.684 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/emulex/ 2023-03-03T06:09:45.685 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/emulex/benet/ 2023-03-03T06:09:45.685 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 290761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-03-03T06:09:45.685 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/ethoc.ko 2023-03-03T06:09:45.685 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/fujitsu/ 2023-03-03T06:09:45.685 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-03-03T06:09:45.685 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ 2023-03-03T06:09:45.685 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 93185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/e100.ko 2023-03-03T06:09:45.685 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/e1000/ 2023-03-03T06:09:45.686 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 300377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-03-03T06:09:45.686 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/e1000e/ 2023-03-03T06:09:45.686 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 588441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-03-03T06:09:45.686 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/fm10k/ 2023-03-03T06:09:45.686 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 250753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-03-03T06:09:45.686 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/i40e/ 2023-03-03T06:09:45.686 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 979137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-03-03T06:09:45.686 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/iavf/ 2023-03-03T06:09:45.687 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 320785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-03-03T06:09:45.687 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/igb/ 2023-03-03T06:09:45.687 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 524993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-03-03T06:09:45.687 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/igbvf/ 2023-03-03T06:09:45.687 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 92905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-03-03T06:09:45.687 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ixgb/ 2023-03-03T06:09:45.687 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 126129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ixgb/ixgb.ko 2023-03-03T06:09:45.688 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-03-03T06:09:45.688 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 747761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-03-03T06:09:45.688 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-03-03T06:09:45.688 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 154305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-03-03T06:09:45.688 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 89905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/jme.ko 2023-03-03T06:09:45.688 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/marvell/ 2023-03-03T06:09:45.688 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-03-03T06:09:45.688 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 106553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-03-03T06:09:45.689 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 124961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-03-03T06:09:45.689 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/ 2023-03-03T06:09:45.689 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-03-03T06:09:45.689 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 745441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-03-03T06:09:45.689 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 277625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-03-03T06:09:45.689 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-03-03T06:09:45.689 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-03-03T06:09:45.689 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1048833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-03-03T06:09:45.690 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/micrel/ 2023-03-03T06:09:45.690 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-03-03T06:09:45.690 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-03-03T06:09:45.690 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-03-03T06:09:45.690 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 95801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-03-03T06:09:45.690 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/myricom/ 2023-03-03T06:09:45.690 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-03-03T06:09:45.691 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 115409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-03-03T06:09:45.691 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/natsemi/ 2023-03-03T06:09:45.691 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 76033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-03-03T06:09:45.691 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-03-03T06:09:45.691 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/neterion/ 2023-03-03T06:09:45.691 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 155121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-03-03T06:09:45.691 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/nvidia/ 2023-03-03T06:09:45.691 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 143961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-03-03T06:09:45.692 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/packetengines/ 2023-03-03T06:09:45.692 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-03-03T06:09:45.692 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-03-03T06:09:45.692 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/qlogic/ 2023-03-03T06:09:45.692 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-03-03T06:09:45.692 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 234649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-03-03T06:09:45.692 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 91945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-03-03T06:09:45.692 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-03-03T06:09:45.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 702361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-03-03T06:09:45.693 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/rdc/ 2023-03-03T06:09:45.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-03-03T06:09:45.693 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/realtek/ 2023-03-03T06:09:45.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 67177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-03-03T06:09:45.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-03-03T06:09:45.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-03-03T06:09:45.693 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 187265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-03-03T06:09:45.694 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/samsung/ 2023-03-03T06:09:45.694 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-03-03T06:09:45.694 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 104737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-03-03T06:09:45.694 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sfc/ 2023-03-03T06:09:45.694 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 848441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-03-03T06:09:45.694 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/silan/ 2023-03-03T06:09:45.694 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-03-03T06:09:45.695 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sis/ 2023-03-03T06:09:45.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-03-03T06:09:45.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 64753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-03-03T06:09:45.695 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/smsc/ 2023-03-03T06:09:45.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-03-03T06:09:45.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 67961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-03-03T06:09:45.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 74297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-03-03T06:09:45.695 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-03-03T06:09:45.696 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/stmicro/ 2023-03-03T06:09:45.696 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-03-03T06:09:45.696 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-03-03T06:09:45.696 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-03-03T06:09:45.696 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-03-03T06:09:45.696 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 476785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-03-03T06:09:45.696 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sun/ 2023-03-03T06:09:45.696 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 106641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-03-03T06:09:45.697 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 172713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sun/niu.ko 2023-03-03T06:09:45.697 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 68657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-03-03T06:09:45.697 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 86489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-03-03T06:09:45.697 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/tehuti/ 2023-03-03T06:09:45.697 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-03-03T06:09:45.697 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/ti/ 2023-03-03T06:09:45.697 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-03-03T06:09:45.698 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/via/ 2023-03-03T06:09:45.698 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 81801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-03-03T06:09:45.698 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-03-03T06:09:45.698 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/wiznet/ 2023-03-03T06:09:45.698 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-03-03T06:09:45.698 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-03-03T06:09:45.698 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/xircom/ 2023-03-03T06:09:45.698 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-03-03T06:09:45.699 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/fddi/ 2023-03-03T06:09:45.699 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/fddi/defxx.ko 2023-03-03T06:09:45.699 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/fddi/skfp/ 2023-03-03T06:09:45.699 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 223521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/fddi/skfp/skfp.ko 2023-03-03T06:09:45.699 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 82889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/geneve.ko 2023-03-03T06:09:45.699 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/hyperv/ 2023-03-03T06:09:45.699 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 238937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-03-03T06:09:45.700 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ieee802154/ 2023-03-03T06:09:45.700 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ieee802154/fakelb.ko 2023-03-03T06:09:45.700 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ifb.ko 2023-03-03T06:09:45.700 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ipvlan/ 2023-03-03T06:09:45.700 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 86849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-03-03T06:09:45.700 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/macvlan.ko 2023-03-03T06:09:45.700 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/macvtap.ko 2023-03-03T06:09:45.700 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/mdio/ 2023-03-03T06:09:45.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-03-03T06:09:45.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-03-03T06:09:45.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/mdio/mdio-gpio.ko 2023-03-03T06:09:45.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/mdio.ko 2023-03-03T06:09:45.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/mii.ko 2023-03-03T06:09:45.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/netconsole.ko 2023-03-03T06:09:45.701 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/nlmon.ko 2023-03-03T06:09:45.701 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/pcs/ 2023-03-03T06:09:45.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-03-03T06:09:45.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-03-03T06:09:45.702 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/ 2023-03-03T06:09:45.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/amd.ko 2023-03-03T06:09:45.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-03-03T06:09:45.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/bcm7xxx.ko 2023-03-03T06:09:45.702 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/bcm87xx.ko 2023-03-03T06:09:45.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/broadcom.ko 2023-03-03T06:09:45.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/cicada.ko 2023-03-03T06:09:45.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/davicom.ko 2023-03-03T06:09:45.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/et1011c.ko 2023-03-03T06:09:45.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/icplus.ko 2023-03-03T06:09:45.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/lxt.ko 2023-03-03T06:09:45.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 75921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/marvell.ko 2023-03-03T06:09:45.703 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/micrel.ko 2023-03-03T06:09:45.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/national.ko 2023-03-03T06:09:45.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 94009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/phylink.ko 2023-03-03T06:09:45.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/qsemi.ko 2023-03-03T06:09:45.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/realtek.ko 2023-03-03T06:09:45.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/smsc.ko 2023-03-03T06:09:45.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/ste10Xp.ko 2023-03-03T06:09:45.704 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/vitesse.ko 2023-03-03T06:09:45.704 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/plip/ 2023-03-03T06:09:45.705 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/plip/plip.ko 2023-03-03T06:09:45.705 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/ 2023-03-03T06:09:45.705 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/bsd_comp.ko 2023-03-03T06:09:45.705 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/ppp_async.ko 2023-03-03T06:09:45.705 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-03-03T06:09:45.705 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-03-03T06:09:45.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-03-03T06:09:45.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/pppoe.ko 2023-03-03T06:09:45.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/pppox.ko 2023-03-03T06:09:45.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/pptp.ko 2023-03-03T06:09:45.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/sb1000.ko 2023-03-03T06:09:45.706 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/slip/ 2023-03-03T06:09:45.706 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/slip/slip.ko 2023-03-03T06:09:45.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/sungem_phy.ko 2023-03-03T06:09:45.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/tap.ko 2023-03-03T06:09:45.707 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/ 2023-03-03T06:09:45.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 105729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/team.ko 2023-03-03T06:09:45.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-03-03T06:09:45.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-03-03T06:09:45.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-03-03T06:09:45.707 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/team_mode_random.ko 2023-03-03T06:09:45.708 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-03-03T06:09:45.708 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 82473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/veth.ko 2023-03-03T06:09:45.708 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/vmxnet3/ 2023-03-03T06:09:45.708 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 121785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-03-03T06:09:45.708 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/vxlan/ 2023-03-03T06:09:45.708 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 251809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/vxlan/vxlan.ko 2023-03-03T06:09:45.708 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/xen-netback/ 2023-03-03T06:09:45.708 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 145569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-03-03T06:09:45.709 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ntb/ 2023-03-03T06:09:45.709 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ntb/ntb.ko 2023-03-03T06:09:45.709 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/nvme/ 2023-03-03T06:09:45.709 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/nvme/host/ 2023-03-03T06:09:45.709 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 260793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/nvme/host/nvme-core.ko 2023-03-03T06:09:45.709 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 105977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/nvme/host/nvme.ko 2023-03-03T06:09:45.709 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/parport/ 2023-03-03T06:09:45.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 130233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/parport/parport.ko 2023-03-03T06:09:45.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/parport/parport_ax88796.ko 2023-03-03T06:09:45.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/parport/parport_cs.ko 2023-03-03T06:09:45.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 84817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/parport/parport_pc.ko 2023-03-03T06:09:45.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/parport/parport_serial.ko 2023-03-03T06:09:45.710 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/ 2023-03-03T06:09:45.710 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/hotplug/ 2023-03-03T06:09:45.710 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-03-03T06:09:45.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-03-03T06:09:45.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-03-03T06:09:45.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/pci-stub.ko 2023-03-03T06:09:45.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/xen-pcifront.ko 2023-03-03T06:09:45.711 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/ 2023-03-03T06:09:45.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/i82092.ko 2023-03-03T06:09:45.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 146321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/pcmcia.ko 2023-03-03T06:09:45.711 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/pcmcia_core.ko 2023-03-03T06:09:45.712 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-03-03T06:09:45.712 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/pd6729.ko 2023-03-03T06:09:45.712 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 107353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/yenta_socket.ko 2023-03-03T06:09:45.712 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/phy/ 2023-03-03T06:09:45.712 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/phy/broadcom/ 2023-03-03T06:09:45.712 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-03-03T06:09:45.712 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/phy/samsung/ 2023-03-03T06:09:45.713 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-03-03T06:09:45.713 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pinctrl/ 2023-03-03T06:09:45.713 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pinctrl/intel/ 2023-03-03T06:09:45.713 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 71145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-03-03T06:09:45.713 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/ 2023-03-03T06:09:45.713 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/ 2023-03-03T06:09:45.713 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-03-03T06:09:45.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros_ec.ko 2023-03-03T06:09:45.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-03-03T06:09:45.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-03-03T06:09:45.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-03-03T06:09:45.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-03-03T06:09:45.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-03-03T06:09:45.714 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/ 2023-03-03T06:09:45.714 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 67201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/acer-wmi.ko 2023-03-03T06:09:45.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/acerhdf.ko 2023-03-03T06:09:45.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/apple-gmux.ko 2023-03-03T06:09:45.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/asus-laptop.ko 2023-03-03T06:09:45.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-03-03T06:09:45.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 110601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/asus-wmi.ko 2023-03-03T06:09:45.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-03-03T06:09:45.715 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-03-03T06:09:45.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-03-03T06:09:45.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-03-03T06:09:45.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-03-03T06:09:45.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/hdaps.ko 2023-03-03T06:09:45.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/ibm_rtl.ko 2023-03-03T06:09:45.716 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/intel/ 2023-03-03T06:09:45.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-03-03T06:09:45.716 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-03-03T06:09:45.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/intel_ips.ko 2023-03-03T06:09:45.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/msi-wmi.ko 2023-03-03T06:09:45.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-03-03T06:09:45.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-03-03T06:09:45.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-03-03T06:09:45.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/samsung-q10.ko 2023-03-03T06:09:45.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-03-03T06:09:45.717 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-03-03T06:09:45.718 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-03-03T06:09:45.718 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-03-03T06:09:45.718 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/wmi.ko 2023-03-03T06:09:45.718 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/ 2023-03-03T06:09:45.718 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/ 2023-03-03T06:09:45.718 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30073 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/88pm860x_battery.ko 2023-03-03T06:09:45.718 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/88pm860x_charger.ko 2023-03-03T06:09:45.719 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/bq2415x_charger.ko 2023-03-03T06:09:45.719 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/bq24190_charger.ko 2023-03-03T06:09:45.719 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/bq24735-charger.ko 2023-03-03T06:09:45.719 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-03-03T06:09:45.719 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/da9030_battery.ko 2023-03-03T06:09:45.719 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/da9052-battery.ko 2023-03-03T06:09:45.719 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/ds2760_battery.ko 2023-03-03T06:09:45.719 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/ds2780_battery.ko 2023-03-03T06:09:45.720 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/ds2781_battery.ko 2023-03-03T06:09:45.720 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/ds2782_battery.ko 2023-03-03T06:09:45.720 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/gpio-charger.ko 2023-03-03T06:09:45.720 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/isp1704_charger.ko 2023-03-03T06:09:45.720 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/lp8727_charger.ko 2023-03-03T06:09:45.720 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/max14577_charger.ko 2023-03-03T06:09:45.720 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/max17040_battery.ko 2023-03-03T06:09:45.720 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/max17042_battery.ko 2023-03-03T06:09:45.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/max8903_charger.ko 2023-03-03T06:09:45.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/max8925_power.ko 2023-03-03T06:09:45.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/pcf50633-charger.ko 2023-03-03T06:09:45.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/pda_power.ko 2023-03-03T06:09:45.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/sbs-battery.ko 2023-03-03T06:09:45.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/test_power.ko 2023-03-03T06:09:45.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/tps65090-charger.ko 2023-03-03T06:09:45.721 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/wm831x_backup.ko 2023-03-03T06:09:45.722 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/wm831x_power.ko 2023-03-03T06:09:45.722 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/wm8350_power.ko 2023-03-03T06:09:45.722 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/powercap/ 2023-03-03T06:09:45.722 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/powercap/intel_rapl_common.ko 2023-03-03T06:09:45.722 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-03-03T06:09:45.722 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pps/ 2023-03-03T06:09:45.722 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pps/clients/ 2023-03-03T06:09:45.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pps/clients/pps-gpio.ko 2023-03-03T06:09:45.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-03-03T06:09:45.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pps/clients/pps_parport.ko 2023-03-03T06:09:45.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pps/pps_core.ko 2023-03-03T06:09:45.723 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ptp/ 2023-03-03T06:09:45.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 64521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ptp/ptp.ko 2023-03-03T06:09:45.723 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ptp/ptp_kvm.ko 2023-03-03T06:09:45.723 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/ 2023-03-03T06:09:45.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-88pm80x.ko 2023-03-03T06:09:45.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-88pm860x.ko 2023-03-03T06:09:45.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-bq32k.ko 2023-03-03T06:09:45.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-bq4802.ko 2023-03-03T06:09:45.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-da9052.ko 2023-03-03T06:09:45.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-da9055.ko 2023-03-03T06:09:45.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-da9063.ko 2023-03-03T06:09:45.724 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1286.ko 2023-03-03T06:09:45.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1307.ko 2023-03-03T06:09:45.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1374.ko 2023-03-03T06:09:45.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1511.ko 2023-03-03T06:09:45.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1553.ko 2023-03-03T06:09:45.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1672.ko 2023-03-03T06:09:45.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1742.ko 2023-03-03T06:09:45.725 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds2404.ko 2023-03-03T06:09:45.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds3232.ko 2023-03-03T06:09:45.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-em3027.ko 2023-03-03T06:09:45.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-fm3130.ko 2023-03-03T06:09:45.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-isl12022.ko 2023-03-03T06:09:45.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-isl1208.ko 2023-03-03T06:09:45.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-lp8788.ko 2023-03-03T06:09:45.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-m41t80.ko 2023-03-03T06:09:45.726 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-m48t35.ko 2023-03-03T06:09:45.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-m48t59.ko 2023-03-03T06:09:45.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-m48t86.ko 2023-03-03T06:09:45.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-max6900.ko 2023-03-03T06:09:45.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-max8907.ko 2023-03-03T06:09:45.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-max8925.ko 2023-03-03T06:09:45.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-max8997.ko 2023-03-03T06:09:45.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-max8998.ko 2023-03-03T06:09:45.727 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-03-03T06:09:45.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-msm6242.ko 2023-03-03T06:09:45.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-palmas.ko 2023-03-03T06:09:45.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-03-03T06:09:45.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-pcf50633.ko 2023-03-03T06:09:45.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-03-03T06:09:45.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-03-03T06:09:45.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-03-03T06:09:45.728 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-03-03T06:09:45.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-rc5t583.ko 2023-03-03T06:09:45.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-03-03T06:09:45.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-03-03T06:09:45.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-03-03T06:09:45.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-rx8025.ko 2023-03-03T06:09:45.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-rx8581.ko 2023-03-03T06:09:45.729 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-s35390a.ko 2023-03-03T06:09:45.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-03-03T06:09:45.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-tps6586x.ko 2023-03-03T06:09:45.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-tps65910.ko 2023-03-03T06:09:45.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-v3020.ko 2023-03-03T06:09:45.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-wm831x.ko 2023-03-03T06:09:45.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-wm8350.ko 2023-03-03T06:09:45.730 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-x1205.ko 2023-03-03T06:09:45.730 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/ 2023-03-03T06:09:45.731 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 80473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/3w-9xxx.ko 2023-03-03T06:09:45.731 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/3w-sas.ko 2023-03-03T06:09:45.731 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/3w-xxxx.ko 2023-03-03T06:09:45.731 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 130513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/BusLogic.ko 2023-03-03T06:09:45.731 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/a100u2w.ko 2023-03-03T06:09:45.731 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aacraid/ 2023-03-03T06:09:45.731 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 278169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-03-03T06:09:45.731 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 102809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/advansys.ko 2023-03-03T06:09:45.732 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aic7xxx/ 2023-03-03T06:09:45.732 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 364593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-03-03T06:09:45.732 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 304337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-03-03T06:09:45.732 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aic94xx/ 2023-03-03T06:09:45.732 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 193489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-03-03T06:09:45.732 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/am53c974.ko 2023-03-03T06:09:45.732 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/arcmsr/ 2023-03-03T06:09:45.732 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 110193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-03-03T06:09:45.733 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/atp870u.ko 2023-03-03T06:09:45.733 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/be2iscsi/ 2023-03-03T06:09:45.733 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 263329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-03-03T06:09:45.733 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/bfa/ 2023-03-03T06:09:45.733 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1178769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/bfa/bfa.ko 2023-03-03T06:09:45.733 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/bnx2fc/ 2023-03-03T06:09:45.733 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 251577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-03-03T06:09:45.734 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/bnx2i/ 2023-03-03T06:09:45.734 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 121057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-03-03T06:09:45.734 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/ch.ko 2023-03-03T06:09:45.734 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/csiostor/ 2023-03-03T06:09:45.734 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 321433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-03-03T06:09:45.734 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/cxgbi/ 2023-03-03T06:09:45.734 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-03-03T06:09:45.735 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 80577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-03-03T06:09:45.735 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-03-03T06:09:45.735 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 108769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-03-03T06:09:45.735 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 153305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-03-03T06:09:45.735 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 78601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/dc395x.ko 2023-03-03T06:09:45.735 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/dmx3191d.ko 2023-03-03T06:09:45.735 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/esas2r/ 2023-03-03T06:09:45.736 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 212161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-03-03T06:09:45.736 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/esp_scsi.ko 2023-03-03T06:09:45.736 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/fcoe/ 2023-03-03T06:09:45.736 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 80761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-03-03T06:09:45.736 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 164201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-03-03T06:09:45.736 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/fdomain.ko 2023-03-03T06:09:45.736 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/fnic/ 2023-03-03T06:09:45.736 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 274297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/fnic/fnic.ko 2023-03-03T06:09:45.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 208345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/hpsa.ko 2023-03-03T06:09:45.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/hptiop.ko 2023-03-03T06:09:45.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/hv_storvsc.ko 2023-03-03T06:09:45.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/imm.ko 2023-03-03T06:09:45.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/initio.ko 2023-03-03T06:09:45.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 302617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/ipr.ko 2023-03-03T06:09:45.737 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 106865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/ips.ko 2023-03-03T06:09:45.738 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/isci/ 2023-03-03T06:09:45.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 328585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/isci/isci.ko 2023-03-03T06:09:45.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-03-03T06:09:45.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/iscsi_tcp.ko 2023-03-03T06:09:45.738 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/libfc/ 2023-03-03T06:09:45.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 325545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/libfc/libfc.ko 2023-03-03T06:09:45.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 147233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/libiscsi.ko 2023-03-03T06:09:45.738 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-03-03T06:09:45.739 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/libsas/ 2023-03-03T06:09:45.739 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 217481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/libsas/libsas.ko 2023-03-03T06:09:45.739 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/lpfc/ 2023-03-03T06:09:45.739 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 2313865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-03-03T06:09:45.739 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/megaraid/ 2023-03-03T06:09:45.739 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 80145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-03-03T06:09:45.739 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-03-03T06:09:45.739 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 324825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-03-03T06:09:45.740 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 86001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/megaraid.ko 2023-03-03T06:09:45.740 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/mpt3sas/ 2023-03-03T06:09:45.740 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 699913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-03-03T06:09:45.740 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/mvsas/ 2023-03-03T06:09:45.740 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 121249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-03-03T06:09:45.740 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 82153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/mvumi.ko 2023-03-03T06:09:45.740 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pcmcia/ 2023-03-03T06:09:45.741 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 82769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-03-03T06:09:45.741 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-03-03T06:09:45.741 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-03-03T06:09:45.741 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-03-03T06:09:45.741 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pm8001/ 2023-03-03T06:09:45.741 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 405633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-03-03T06:09:45.741 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 131457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pmcraid.ko 2023-03-03T06:09:45.741 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/ppa.ko 2023-03-03T06:09:45.742 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 57137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qla1280.ko 2023-03-03T06:09:45.742 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qla2xxx/ 2023-03-03T06:09:45.742 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1854193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-03-03T06:09:45.742 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 89473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-03-03T06:09:45.742 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qla4xxx/ 2023-03-03T06:09:45.742 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 596697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-03-03T06:09:45.742 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qlogicfas408.ko 2023-03-03T06:09:45.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/raid_class.ko 2023-03-03T06:09:45.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 246129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/scsi_debug.ko 2023-03-03T06:09:45.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 173377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-03-03T06:09:45.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 276017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-03-03T06:09:45.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 94793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-03-03T06:09:45.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 73665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-03-03T06:09:45.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-03-03T06:09:45.743 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/ses.ko 2023-03-03T06:09:45.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 116161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/st.ko 2023-03-03T06:09:45.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/stex.ko 2023-03-03T06:09:45.744 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/sym53c8xx_2/ 2023-03-03T06:09:45.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 167465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-03-03T06:09:45.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/virtio_scsi.ko 2023-03-03T06:09:45.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-03-03T06:09:45.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/wd719x.ko 2023-03-03T06:09:45.744 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/xen-scsifront.ko 2023-03-03T06:09:45.745 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ssb/ 2023-03-03T06:09:45.745 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 130601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ssb/ssb.ko 2023-03-03T06:09:45.745 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/ 2023-03-03T06:09:45.745 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/iscsi/ 2023-03-03T06:09:45.745 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 761009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-03-03T06:09:45.745 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/loopback/ 2023-03-03T06:09:45.745 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/loopback/tcm_loop.ko 2023-03-03T06:09:45.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/target_core_file.ko 2023-03-03T06:09:45.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/target_core_iblock.ko 2023-03-03T06:09:45.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 937993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/target_core_mod.ko 2023-03-03T06:09:45.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/target_core_pscsi.ko 2023-03-03T06:09:45.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 124425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/target_core_user.ko 2023-03-03T06:09:45.746 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/tcm_fc/ 2023-03-03T06:09:45.746 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 77081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-03-03T06:09:45.746 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/ 2023-03-03T06:09:45.747 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/ 2023-03-03T06:09:45.747 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/ 2023-03-03T06:09:45.747 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-03-03T06:09:45.747 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-03-03T06:09:45.747 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-03-03T06:09:45.747 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-03-03T06:09:45.747 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-03-03T06:09:45.747 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-03-03T06:09:45.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-03-03T06:09:45.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-03-03T06:09:45.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-03-03T06:09:45.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-03-03T06:09:45.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-03-03T06:09:45.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-03-03T06:09:45.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/intel_menlow.ko 2023-03-03T06:09:45.748 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-03-03T06:09:45.749 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-03-03T06:09:45.749 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-03-03T06:09:45.749 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-03-03T06:09:45.749 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/ 2023-03-03T06:09:45.749 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/ipwireless/ 2023-03-03T06:09:45.749 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 80145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-03-03T06:09:45.749 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/moxa.ko 2023-03-03T06:09:45.750 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/mxser.ko 2023-03-03T06:09:45.750 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 103217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/n_gsm.ko 2023-03-03T06:09:45.750 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/n_hdlc.ko 2023-03-03T06:09:45.750 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 68865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/nozomi.ko 2023-03-03T06:09:45.750 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/ 2023-03-03T06:09:45.750 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/8250/ 2023-03-03T06:09:45.750 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-03-03T06:09:45.750 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-03-03T06:09:45.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-03-03T06:09:45.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/altera_uart.ko 2023-03-03T06:09:45.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/arc_uart.ko 2023-03-03T06:09:45.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-03-03T06:09:45.751 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/jsm/ 2023-03-03T06:09:45.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 102929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-03-03T06:09:45.751 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/rp2.ko 2023-03-03T06:09:45.752 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/sc16is7xx.ko 2023-03-03T06:09:45.752 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 106785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/synclink_gt.ko 2023-03-03T06:09:45.752 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ufs/ 2023-03-03T06:09:45.752 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ufs/core/ 2023-03-03T06:09:45.752 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 507913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-03-03T06:09:45.752 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ufs/host/ 2023-03-03T06:09:45.752 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-03-03T06:09:45.752 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-03-03T06:09:45.753 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/ 2023-03-03T06:09:45.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio.ko 2023-03-03T06:09:45.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_aec.ko 2023-03-03T06:09:45.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_cif.ko 2023-03-03T06:09:45.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_dmem_genirq.ko 2023-03-03T06:09:45.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_mf624.ko 2023-03-03T06:09:45.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_netx.ko 2023-03-03T06:09:45.753 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_pci_generic.ko 2023-03-03T06:09:45.754 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-03-03T06:09:45.754 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_sercos3.ko 2023-03-03T06:09:45.754 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/ 2023-03-03T06:09:45.754 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/c67x00/ 2023-03-03T06:09:45.754 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 73801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/c67x00/c67x00.ko 2023-03-03T06:09:45.754 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/chipidea/ 2023-03-03T06:09:45.754 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 148737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-03-03T06:09:45.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-03-03T06:09:45.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-03-03T06:09:45.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-03-03T06:09:45.755 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/class/ 2023-03-03T06:09:45.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 86289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/class/cdc-acm.ko 2023-03-03T06:09:45.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/class/cdc-wdm.ko 2023-03-03T06:09:45.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/class/usblp.ko 2023-03-03T06:09:45.755 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 93297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/class/usbtmc.ko 2023-03-03T06:09:45.756 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/common/ 2023-03-03T06:09:45.756 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/common/ulpi.ko 2023-03-03T06:09:45.756 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/dwc2/ 2023-03-03T06:09:45.756 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-03-03T06:09:45.756 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/dwc3/ 2023-03-03T06:09:45.756 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-03-03T06:09:45.756 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-03-03T06:09:45.756 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 308233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/dwc3/dwc3.ko 2023-03-03T06:09:45.757 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/ 2023-03-03T06:09:45.757 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/ 2023-03-03T06:09:45.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/u_ether.ko 2023-03-03T06:09:45.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/u_serial.ko 2023-03-03T06:09:45.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-03-03T06:09:45.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-03-03T06:09:45.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-03-03T06:09:45.757 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-03-03T06:09:45.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 92065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-03-03T06:09:45.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-03-03T06:09:45.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 121233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-03-03T06:09:45.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-03-03T06:09:45.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-03-03T06:09:45.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-03-03T06:09:45.758 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-03-03T06:09:45.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-03-03T06:09:45.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-03-03T06:09:45.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-03-03T06:09:45.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 72505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-03-03T06:09:45.759 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/ 2023-03-03T06:09:45.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-03-03T06:09:45.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-03-03T06:09:45.759 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-03-03T06:09:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-03-03T06:09:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-03-03T06:09:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-03-03T06:09:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-03-03T06:09:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-03-03T06:09:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-03-03T06:09:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-03-03T06:09:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-03-03T06:09:45.761 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-03-03T06:09:45.761 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 62145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-03-03T06:09:45.761 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-03-03T06:09:45.761 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 144601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/libcomposite.ko 2023-03-03T06:09:45.761 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/ 2023-03-03T06:09:45.761 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-03-03T06:09:45.761 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/bdc/ 2023-03-03T06:09:45.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 158873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-03-03T06:09:45.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-03-03T06:09:45.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-03-03T06:09:45.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-03-03T06:09:45.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 57921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-03-03T06:09:45.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 75705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/net2272.ko 2023-03-03T06:09:45.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 77905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/net2280.ko 2023-03-03T06:09:45.762 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 74081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-03-03T06:09:45.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-03-03T06:09:45.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-03-03T06:09:45.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 80881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-03-03T06:09:45.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 149737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-03-03T06:09:45.763 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/ 2023-03-03T06:09:45.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/bcma-hcd.ko 2023-03-03T06:09:45.763 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 80897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/isp116x-hcd.ko 2023-03-03T06:09:45.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 88897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-03-03T06:09:45.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 81417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-03-03T06:09:45.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/sl811-hcd.ko 2023-03-03T06:09:45.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/sl811_cs.ko 2023-03-03T06:09:45.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/ssb-hcd.ko 2023-03-03T06:09:45.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 98497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/u132-hcd.ko 2023-03-03T06:09:45.764 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-03-03T06:09:45.764 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/image/ 2023-03-03T06:09:45.765 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/image/mdc800.ko 2023-03-03T06:09:45.765 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/image/microtek.ko 2023-03-03T06:09:45.765 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/ 2023-03-03T06:09:45.765 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/adutux.ko 2023-03-03T06:09:45.765 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/appledisplay.ko 2023-03-03T06:09:45.765 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-03-03T06:09:45.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/cytherm.ko 2023-03-03T06:09:45.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/ehset.ko 2023-03-03T06:09:45.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/emi26.ko 2023-03-03T06:09:45.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/emi62.ko 2023-03-03T06:09:45.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/ezusb.ko 2023-03-03T06:09:45.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 78945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/ftdi-elan.ko 2023-03-03T06:09:45.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/idmouse.ko 2023-03-03T06:09:45.766 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/iowarrior.ko 2023-03-03T06:09:45.767 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/isight_firmware.ko 2023-03-03T06:09:45.767 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/ldusb.ko 2023-03-03T06:09:45.767 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/legousbtower.ko 2023-03-03T06:09:45.767 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/lvstest.ko 2023-03-03T06:09:45.767 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/sisusbvga/ 2023-03-03T06:09:45.767 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 64617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-03-03T06:09:45.767 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/trancevibrator.ko 2023-03-03T06:09:45.767 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/usb3503.ko 2023-03-03T06:09:45.768 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/usblcd.ko 2023-03-03T06:09:45.768 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/usbsevseg.ko 2023-03-03T06:09:45.768 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/usbtest.ko 2023-03-03T06:09:45.768 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/uss720.ko 2023-03-03T06:09:45.768 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/yurex.ko 2023-03-03T06:09:45.768 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/mon/ 2023-03-03T06:09:45.768 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 71345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/mon/usbmon.ko 2023-03-03T06:09:45.769 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/musb/ 2023-03-03T06:09:45.769 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 303201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/musb/musb_hdrc.ko 2023-03-03T06:09:45.769 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/phy/ 2023-03-03T06:09:45.769 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/phy/phy-generic.ko 2023-03-03T06:09:45.769 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-03-03T06:09:45.769 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/phy/phy-isp1301.ko 2023-03-03T06:09:45.769 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/phy/phy-tahvo.ko 2023-03-03T06:09:45.769 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ 2023-03-03T06:09:45.770 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/aircable.ko 2023-03-03T06:09:45.770 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ark3116.ko 2023-03-03T06:09:45.770 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/belkin_sa.ko 2023-03-03T06:09:45.770 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ch341.ko 2023-03-03T06:09:45.770 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/cp210x.ko 2023-03-03T06:09:45.770 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/cyberjack.ko 2023-03-03T06:09:45.770 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/cypress_m8.ko 2023-03-03T06:09:45.770 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-03-03T06:09:45.771 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/empeg.ko 2023-03-03T06:09:45.771 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/f81232.ko 2023-03-03T06:09:45.771 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 176201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-03-03T06:09:45.771 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/garmin_gps.ko 2023-03-03T06:09:45.771 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 138737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/io_edgeport.ko 2023-03-03T06:09:45.771 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 127865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/io_ti.ko 2023-03-03T06:09:45.771 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 67625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ipaq.ko 2023-03-03T06:09:45.771 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ipw.ko 2023-03-03T06:09:45.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ir-usb.ko 2023-03-03T06:09:45.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-03-03T06:09:45.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 109001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/keyspan.ko 2023-03-03T06:09:45.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-03-03T06:09:45.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-03-03T06:09:45.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/kobil_sct.ko 2023-03-03T06:09:45.772 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/mct_u232.ko 2023-03-03T06:09:45.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/metro-usb.ko 2023-03-03T06:09:45.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/mos7720.ko 2023-03-03T06:09:45.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 78345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/mos7840.ko 2023-03-03T06:09:45.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 58217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/mxuport.ko 2023-03-03T06:09:45.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/navman.ko 2023-03-03T06:09:45.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/omninet.ko 2023-03-03T06:09:45.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/opticon.ko 2023-03-03T06:09:45.773 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 186489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/option.ko 2023-03-03T06:09:45.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/oti6858.ko 2023-03-03T06:09:45.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/pl2303.ko 2023-03-03T06:09:45.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/qcaux.ko 2023-03-03T06:09:45.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/qcserial.ko 2023-03-03T06:09:45.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/quatech2.ko 2023-03-03T06:09:45.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/safe_serial.ko 2023-03-03T06:09:45.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/sierra.ko 2023-03-03T06:09:45.774 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/spcp8x5.ko 2023-03-03T06:09:45.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ssu100.ko 2023-03-03T06:09:45.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/symbolserial.ko 2023-03-03T06:09:45.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-03-03T06:09:45.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-03-03T06:09:45.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/usb_debug.ko 2023-03-03T06:09:45.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/usb_wwan.ko 2023-03-03T06:09:45.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 123553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/usbserial.ko 2023-03-03T06:09:45.775 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/visor.ko 2023-03-03T06:09:45.776 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/whiteheat.ko 2023-03-03T06:09:45.776 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/wishbone-serial.ko 2023-03-03T06:09:45.776 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/xsens_mt.ko 2023-03-03T06:09:45.776 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ 2023-03-03T06:09:45.776 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/uas.ko 2023-03-03T06:09:45.776 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-alauda.ko 2023-03-03T06:09:45.776 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-cypress.ko 2023-03-03T06:09:45.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-datafab.ko 2023-03-03T06:09:45.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-03-03T06:09:45.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-freecom.ko 2023-03-03T06:09:45.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-isd200.ko 2023-03-03T06:09:45.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-03-03T06:09:45.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-karma.ko 2023-03-03T06:09:45.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-03-03T06:09:45.777 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-realtek.ko 2023-03-03T06:09:45.778 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-03-03T06:09:45.778 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-03-03T06:09:45.778 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-usbat.ko 2023-03-03T06:09:45.778 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 177089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/usb-storage.ko 2023-03-03T06:09:45.778 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/usbip/ 2023-03-03T06:09:45.778 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 66673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/usbip/usbip-core.ko 2023-03-03T06:09:45.778 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 73201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/usbip/usbip-host.ko 2023-03-03T06:09:45.779 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 108825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-03-03T06:09:45.779 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vfio/ 2023-03-03T06:09:45.779 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vfio/pci/ 2023-03-03T06:09:45.779 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 147089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-03-03T06:09:45.779 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-03-03T06:09:45.779 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 94817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vfio/vfio.ko 2023-03-03T06:09:45.779 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 72001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-03-03T06:09:45.779 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vhost/ 2023-03-03T06:09:45.780 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 98497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vhost/vhost.ko 2023-03-03T06:09:45.780 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vhost/vhost_iotlb.ko 2023-03-03T06:09:45.780 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 57905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vhost/vhost_net.ko 2023-03-03T06:09:45.780 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 91265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vhost/vhost_scsi.ko 2023-03-03T06:09:45.780 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/ 2023-03-03T06:09:45.780 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/ 2023-03-03T06:09:45.780 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-03-03T06:09:45.780 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/aat2870_bl.ko 2023-03-03T06:09:45.781 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/adp5520_bl.ko 2023-03-03T06:09:45.781 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/adp8860_bl.ko 2023-03-03T06:09:45.781 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/adp8870_bl.ko 2023-03-03T06:09:45.781 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/apple_bl.ko 2023-03-03T06:09:45.781 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/as3711_bl.ko 2023-03-03T06:09:45.781 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/bd6107.ko 2023-03-03T06:09:45.781 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/da903x_bl.ko 2023-03-03T06:09:45.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/da9052_bl.ko 2023-03-03T06:09:45.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/gpio_backlight.ko 2023-03-03T06:09:45.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/kb3886_bl.ko 2023-03-03T06:09:45.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/lcd.ko 2023-03-03T06:09:45.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/lm3533_bl.ko 2023-03-03T06:09:45.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/lm3639_bl.ko 2023-03-03T06:09:45.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/lv5207lp.ko 2023-03-03T06:09:45.782 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/max8925_bl.ko 2023-03-03T06:09:45.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/pandora_bl.ko 2023-03-03T06:09:45.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-03-03T06:09:45.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/platform_lcd.ko 2023-03-03T06:09:45.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/wm831x_bl.ko 2023-03-03T06:09:45.783 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/ 2023-03-03T06:09:45.783 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/masters/ 2023-03-03T06:09:45.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/masters/ds1wm.ko 2023-03-03T06:09:45.783 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/masters/ds2482.ko 2023-03-03T06:09:45.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 39569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/masters/ds2490.ko 2023-03-03T06:09:45.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/masters/matrox_w1.ko 2023-03-03T06:09:45.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/masters/w1-gpio.ko 2023-03-03T06:09:45.784 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/ 2023-03-03T06:09:45.784 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-03-03T06:09:45.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-03-03T06:09:45.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-03-03T06:09:45.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-03-03T06:09:45.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-03-03T06:09:45.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-03-03T06:09:45.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-03-03T06:09:45.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-03-03T06:09:45.785 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-03-03T06:09:45.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_smem.ko 2023-03-03T06:09:45.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_therm.ko 2023-03-03T06:09:45.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 102657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/wire.ko 2023-03-03T06:09:45.786 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/ 2023-03-03T06:09:45.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/acquirewdt.ko 2023-03-03T06:09:45.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/advantechwdt.ko 2023-03-03T06:09:45.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-03-03T06:09:45.786 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-03-03T06:09:45.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/cpu5wdt.ko 2023-03-03T06:09:45.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/da9052_wdt.ko 2023-03-03T06:09:45.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/da9055_wdt.ko 2023-03-03T06:09:45.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/da9063_wdt.ko 2023-03-03T06:09:45.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/dw_wdt.ko 2023-03-03T06:09:45.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/eurotechwdt.ko 2023-03-03T06:09:45.787 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-03-03T06:09:45.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/hpwdt.ko 2023-03-03T06:09:45.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/i6300esb.ko 2023-03-03T06:09:45.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-03-03T06:09:45.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-03-03T06:09:45.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/ib700wdt.ko 2023-03-03T06:09:45.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/ibmasr.ko 2023-03-03T06:09:45.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-03-03T06:09:45.788 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-03-03T06:09:45.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/it87_wdt.ko 2023-03-03T06:09:45.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/kempld_wdt.ko 2023-03-03T06:09:45.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/machzwd.ko 2023-03-03T06:09:45.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/mena21_wdt.ko 2023-03-03T06:09:45.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-03-03T06:09:45.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/nv_tco.ko 2023-03-03T06:09:45.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-03-03T06:09:45.789 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/pc87413_wdt.ko 2023-03-03T06:09:45.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/pcwd_pci.ko 2023-03-03T06:09:45.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/pcwd_usb.ko 2023-03-03T06:09:45.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/retu_wdt.ko 2023-03-03T06:09:45.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-03-03T06:09:45.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-03-03T06:09:45.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-03-03T06:09:45.790 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/sc1200wdt.ko 2023-03-03T06:09:45.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-03-03T06:09:45.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-03-03T06:09:45.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/softdog.ko 2023-03-03T06:09:45.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/sp5100_tco.ko 2023-03-03T06:09:45.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/twl4030_wdt.ko 2023-03-03T06:09:45.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/via_wdt.ko 2023-03-03T06:09:45.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-03-03T06:09:45.791 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-03-03T06:09:45.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-03-03T06:09:45.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/wafer5823wdt.ko 2023-03-03T06:09:45.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/wdt_pci.ko 2023-03-03T06:09:45.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/wm831x_wdt.ko 2023-03-03T06:09:45.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/wm8350_wdt.ko 2023-03-03T06:09:45.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/xen_wdt.ko 2023-03-03T06:09:45.792 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/ 2023-03-03T06:09:45.792 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-evtchn.ko 2023-03-03T06:09:45.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-gntalloc.ko 2023-03-03T06:09:45.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-gntdev.ko 2023-03-03T06:09:45.793 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-pciback/ 2023-03-03T06:09:45.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 179209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-03-03T06:09:45.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-privcmd.ko 2023-03-03T06:09:45.793 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 74009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-scsiback.ko 2023-03-03T06:09:45.793 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xenfs/ 2023-03-03T06:09:45.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xenfs/xenfs.ko 2023-03-03T06:09:45.794 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ 2023-03-03T06:09:45.794 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/9p/ 2023-03-03T06:09:45.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 170937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/9p/9p.ko 2023-03-03T06:09:45.794 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/adfs/ 2023-03-03T06:09:45.794 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/adfs/adfs.ko 2023-03-03T06:09:45.794 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/affs/ 2023-03-03T06:09:45.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 162665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/affs/affs.ko 2023-03-03T06:09:45.795 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/afs/ 2023-03-03T06:09:45.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 886529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/afs/kafs.ko 2023-03-03T06:09:45.795 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/autofs/ 2023-03-03T06:09:45.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 100825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/autofs/autofs4.ko 2023-03-03T06:09:45.795 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/befs/ 2023-03-03T06:09:45.795 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 64073 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/befs/befs.ko 2023-03-03T06:09:45.795 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/bfs/ 2023-03-03T06:09:45.796 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/bfs/bfs.ko 2023-03-03T06:09:45.796 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/binfmt_misc.ko 2023-03-03T06:09:45.796 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/btrfs/ 2023-03-03T06:09:45.796 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 3478321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/btrfs/btrfs.ko 2023-03-03T06:09:45.796 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/cachefiles/ 2023-03-03T06:09:45.796 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 293745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/cachefiles/cachefiles.ko 2023-03-03T06:09:45.796 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ceph/ 2023-03-03T06:09:45.797 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1240393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ceph/ceph.ko 2023-03-03T06:09:45.797 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/cifs/ 2023-03-03T06:09:45.797 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 2600641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/cifs/cifs.ko 2023-03-03T06:09:45.797 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/coda/ 2023-03-03T06:09:45.797 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 99745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/coda/coda.ko 2023-03-03T06:09:45.797 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/configfs/ 2023-03-03T06:09:45.797 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 105777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/configfs/configfs.ko 2023-03-03T06:09:45.797 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/cramfs/ 2023-03-03T06:09:45.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/cramfs/cramfs.ko 2023-03-03T06:09:45.798 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/dlm/ 2023-03-03T06:09:45.798 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 637665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/dlm/dlm.ko 2023-03-03T06:09:45.827 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/efs/ 2023-03-03T06:09:45.827 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/efs/efs.ko 2023-03-03T06:09:45.827 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/f2fs/ 2023-03-03T06:09:45.827 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1485049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/f2fs/f2fs.ko 2023-03-03T06:09:45.827 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/fat/ 2023-03-03T06:09:45.828 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/fat/msdos.ko 2023-03-03T06:09:45.828 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/freevxfs/ 2023-03-03T06:09:45.828 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/freevxfs/freevxfs.ko 2023-03-03T06:09:45.828 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/fscache/ 2023-03-03T06:09:45.828 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 204017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/fscache/fscache.ko 2023-03-03T06:09:45.828 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/fuse/ 2023-03-03T06:09:45.828 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/fuse/cuse.ko 2023-03-03T06:09:45.828 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/gfs2/ 2023-03-03T06:09:45.829 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1234073 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/gfs2/gfs2.ko 2023-03-03T06:09:45.829 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/hfs/ 2023-03-03T06:09:45.829 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 144753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/hfs/hfs.ko 2023-03-03T06:09:45.829 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/hfsplus/ 2023-03-03T06:09:45.829 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 229193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/hfsplus/hfsplus.ko 2023-03-03T06:09:45.829 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/hpfs/ 2023-03-03T06:09:45.829 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 200097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/hpfs/hpfs.ko 2023-03-03T06:09:45.830 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/isofs/ 2023-03-03T06:09:45.830 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 77113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/isofs/isofs.ko 2023-03-03T06:09:45.830 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/jfs/ 2023-03-03T06:09:45.830 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 424905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/jfs/jfs.ko 2023-03-03T06:09:45.830 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/lockd/ 2023-03-03T06:09:45.830 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 279945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/lockd/lockd.ko 2023-03-03T06:09:45.830 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/minix/ 2023-03-03T06:09:45.830 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 88993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/minix/minix.ko 2023-03-03T06:09:45.831 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/netfs/ 2023-03-03T06:09:45.831 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 113025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/netfs/netfs.ko 2023-03-03T06:09:45.831 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/ 2023-03-03T06:09:45.831 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/blocklayout/ 2023-03-03T06:09:45.831 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 89729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-03-03T06:09:45.831 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/filelayout/ 2023-03-03T06:09:45.831 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-03-03T06:09:45.832 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/flexfilelayout/ 2023-03-03T06:09:45.832 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 125041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-03-03T06:09:45.832 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1072689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/nfs.ko 2023-03-03T06:09:45.832 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/nfsv2.ko 2023-03-03T06:09:45.832 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 124673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/nfsv3.ko 2023-03-03T06:09:45.832 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 2057697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/nfsv4.ko 2023-03-03T06:09:45.832 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs_common/ 2023-03-03T06:09:45.832 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs_common/grace.ko 2023-03-03T06:09:45.833 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs_common/nfs_acl.ko 2023-03-03T06:09:45.833 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfsd/ 2023-03-03T06:09:45.833 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1585681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfsd/nfsd.ko 2023-03-03T06:09:45.833 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nilfs2/ 2023-03-03T06:09:45.833 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 586265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nilfs2/nilfs2.ko 2023-03-03T06:09:45.833 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/ 2023-03-03T06:09:45.833 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-celtic.ko 2023-03-03T06:09:45.833 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-centeuro.ko 2023-03-03T06:09:45.834 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-croatian.ko 2023-03-03T06:09:45.834 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-cyrillic.ko 2023-03-03T06:09:45.834 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-gaelic.ko 2023-03-03T06:09:45.834 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-greek.ko 2023-03-03T06:09:45.834 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-iceland.ko 2023-03-03T06:09:45.834 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-inuit.ko 2023-03-03T06:09:45.834 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-roman.ko 2023-03-03T06:09:45.835 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-romanian.ko 2023-03-03T06:09:45.835 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-turkish.ko 2023-03-03T06:09:45.835 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_ascii.ko 2023-03-03T06:09:45.835 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp1250.ko 2023-03-03T06:09:45.835 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp1251.ko 2023-03-03T06:09:45.835 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp1255.ko 2023-03-03T06:09:45.835 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp737.ko 2023-03-03T06:09:45.835 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp775.ko 2023-03-03T06:09:45.836 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp850.ko 2023-03-03T06:09:45.836 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp852.ko 2023-03-03T06:09:45.836 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp855.ko 2023-03-03T06:09:45.836 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp857.ko 2023-03-03T06:09:45.836 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp860.ko 2023-03-03T06:09:45.836 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp861.ko 2023-03-03T06:09:45.836 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp862.ko 2023-03-03T06:09:45.836 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp863.ko 2023-03-03T06:09:45.837 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp864.ko 2023-03-03T06:09:45.837 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp865.ko 2023-03-03T06:09:45.837 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp866.ko 2023-03-03T06:09:45.837 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp869.ko 2023-03-03T06:09:45.837 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp874.ko 2023-03-03T06:09:45.837 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 93649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp932.ko 2023-03-03T06:09:45.837 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 142185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp936.ko 2023-03-03T06:09:45.838 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 165321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp949.ko 2023-03-03T06:09:45.838 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 117625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp950.ko 2023-03-03T06:09:45.838 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_euc-jp.ko 2023-03-03T06:09:45.838 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-1.ko 2023-03-03T06:09:45.838 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-13.ko 2023-03-03T06:09:45.838 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-14.ko 2023-03-03T06:09:45.838 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-15.ko 2023-03-03T06:09:45.838 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-2.ko 2023-03-03T06:09:45.839 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-3.ko 2023-03-03T06:09:45.839 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-4.ko 2023-03-03T06:09:45.839 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-5.ko 2023-03-03T06:09:45.839 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-6.ko 2023-03-03T06:09:45.839 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-7.ko 2023-03-03T06:09:45.839 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-9.ko 2023-03-03T06:09:45.839 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_koi8-r.ko 2023-03-03T06:09:45.839 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_koi8-ru.ko 2023-03-03T06:09:45.840 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_koi8-u.ko 2023-03-03T06:09:45.840 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_utf8.ko 2023-03-03T06:09:45.840 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ntfs/ 2023-03-03T06:09:45.840 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 219449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ntfs/ntfs.ko 2023-03-03T06:09:45.840 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/ 2023-03-03T06:09:45.840 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/cluster/ 2023-03-03T06:09:45.840 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 242329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-03-03T06:09:45.841 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/dlm/ 2023-03-03T06:09:45.841 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 439761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-03-03T06:09:45.841 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/dlmfs/ 2023-03-03T06:09:45.841 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-03-03T06:09:45.841 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 2625897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/ocfs2.ko 2023-03-03T06:09:45.841 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-03-03T06:09:45.841 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-03-03T06:09:45.841 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-03-03T06:09:45.842 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/omfs/ 2023-03-03T06:09:45.842 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 53105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/omfs/omfs.ko 2023-03-03T06:09:45.842 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/overlayfs/ 2023-03-03T06:09:45.842 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 334953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/overlayfs/overlay.ko 2023-03-03T06:09:45.842 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/pstore/ 2023-03-03T06:09:45.842 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/pstore/ramoops.ko 2023-03-03T06:09:45.842 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/qnx4/ 2023-03-03T06:09:45.843 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/qnx4/qnx4.ko 2023-03-03T06:09:45.843 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/qnx6/ 2023-03-03T06:09:45.843 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/qnx6/qnx6.ko 2023-03-03T06:09:45.843 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/quota/ 2023-03-03T06:09:45.843 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/quota/quota_tree.ko 2023-03-03T06:09:45.843 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/quota/quota_v1.ko 2023-03-03T06:09:45.843 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/quota/quota_v2.ko 2023-03-03T06:09:45.843 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/reiserfs/ 2023-03-03T06:09:45.844 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 530121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/reiserfs/reiserfs.ko 2023-03-03T06:09:45.844 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/romfs/ 2023-03-03T06:09:45.844 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/romfs/romfs.ko 2023-03-03T06:09:45.844 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/smbfs_common/ 2023-03-03T06:09:45.844 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/smbfs_common/cifs_arc4.ko 2023-03-03T06:09:45.844 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/smbfs_common/cifs_md4.ko 2023-03-03T06:09:45.844 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/squashfs/ 2023-03-03T06:09:45.844 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 134969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/squashfs/squashfs.ko 2023-03-03T06:09:45.845 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/sysv/ 2023-03-03T06:09:45.845 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 92681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/sysv/sysv.ko 2023-03-03T06:09:45.845 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/udf/ 2023-03-03T06:09:45.845 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 253265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/udf/udf.ko 2023-03-03T06:09:45.845 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ufs/ 2023-03-03T06:09:45.845 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 162185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ufs/ufs.ko 2023-03-03T06:09:45.845 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/xfs/ 2023-03-03T06:09:45.845 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 3960561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/xfs/xfs.ko 2023-03-03T06:09:45.846 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/kernel/ 2023-03-03T06:09:45.846 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/kernel/locking/ 2023-03-03T06:09:45.846 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/kernel/locking/locktorture.ko 2023-03-03T06:09:45.846 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/kernel/time/ 2023-03-03T06:09:45.846 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/kernel/time/test_udelay.ko 2023-03-03T06:09:45.846 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/kernel/torture.ko 2023-03-03T06:09:45.846 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/ 2023-03-03T06:09:45.847 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/crc-itu-t.ko 2023-03-03T06:09:45.847 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 5729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/crc7.ko 2023-03-03T06:09:45.847 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/crc8.ko 2023-03-03T06:09:45.847 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/crypto/ 2023-03-03T06:09:45.847 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/crypto/libarc4.ko 2023-03-03T06:09:45.847 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/crypto/libdes.ko 2023-03-03T06:09:45.847 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/interval_tree_test.ko 2023-03-03T06:09:45.847 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/libcrc32c.ko 2023-03-03T06:09:45.848 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/lru_cache.ko 2023-03-03T06:09:45.848 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/lz4/ 2023-03-03T06:09:45.848 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/lz4/lz4_compress.ko 2023-03-03T06:09:45.848 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/lz4/lz4hc_compress.ko 2023-03-03T06:09:45.848 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/math/ 2023-03-03T06:09:45.848 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 5897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/math/cordic.ko 2023-03-03T06:09:45.848 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/memory-notifier-error-inject.ko 2023-03-03T06:09:45.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/notifier-error-inject.ko 2023-03-03T06:09:45.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/percpu_test.ko 2023-03-03T06:09:45.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/pm-notifier-error-inject.ko 2023-03-03T06:09:45.849 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/raid6/ 2023-03-03T06:09:45.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 141425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/raid6/raid6_pq.ko 2023-03-03T06:09:45.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/rbtree_test.ko 2023-03-03T06:09:45.849 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/reed_solomon/ 2023-03-03T06:09:45.849 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/reed_solomon/reed_solomon.ko 2023-03-03T06:09:45.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/test-kstrtox.ko 2023-03-03T06:09:45.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/test-string_helpers.ko 2023-03-03T06:09:45.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 4759153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/test_bpf.ko 2023-03-03T06:09:45.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/test_firmware.ko 2023-03-03T06:09:45.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 5873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/test_module.ko 2023-03-03T06:09:45.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/test_user_copy.ko 2023-03-03T06:09:45.850 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/ts_bm.ko 2023-03-03T06:09:45.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/ts_fsm.ko 2023-03-03T06:09:45.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/ts_kmp.ko 2023-03-03T06:09:45.851 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/xz/ 2023-03-03T06:09:45.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/xz/xz_dec_test.ko 2023-03-03T06:09:45.851 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/mm/ 2023-03-03T06:09:45.851 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/mm/hwpoison-inject.ko 2023-03-03T06:09:45.851 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ 2023-03-03T06:09:45.851 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/ 2023-03-03T06:09:45.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/6lowpan.ko 2023-03-03T06:09:45.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_dest.ko 2023-03-03T06:09:45.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_fragment.ko 2023-03-03T06:09:45.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_hop.ko 2023-03-03T06:09:45.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_ipv6.ko 2023-03-03T06:09:45.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_mobility.ko 2023-03-03T06:09:45.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_routing.ko 2023-03-03T06:09:45.852 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_udp.ko 2023-03-03T06:09:45.853 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/802/ 2023-03-03T06:09:45.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/802/garp.ko 2023-03-03T06:09:45.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/802/mrp.ko 2023-03-03T06:09:45.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/802/p8022.ko 2023-03-03T06:09:45.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/802/psnap.ko 2023-03-03T06:09:45.853 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/802/stp.ko 2023-03-03T06:09:45.853 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/8021q/ 2023-03-03T06:09:45.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 86249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/8021q/8021q.ko 2023-03-03T06:09:45.854 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/9p/ 2023-03-03T06:09:45.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 178153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/9p/9pnet.ko 2023-03-03T06:09:45.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/9p/9pnet_fd.ko 2023-03-03T06:09:45.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/9p/9pnet_rdma.ko 2023-03-03T06:09:45.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/9p/9pnet_virtio.ko 2023-03-03T06:09:45.854 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/batman-adv/ 2023-03-03T06:09:45.854 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 651577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/batman-adv/batman-adv.ko 2023-03-03T06:09:45.855 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/ 2023-03-03T06:09:45.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 84849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/br_netfilter.ko 2023-03-03T06:09:45.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 755113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/bridge.ko 2023-03-03T06:09:45.855 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ 2023-03-03T06:09:45.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-03-03T06:09:45.855 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_among.ko 2023-03-03T06:09:45.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-03-03T06:09:45.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-03-03T06:09:45.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-03-03T06:09:45.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-03-03T06:09:45.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-03-03T06:09:45.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-03-03T06:09:45.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_log.ko 2023-03-03T06:09:45.856 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-03-03T06:09:45.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-03-03T06:09:45.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-03-03T06:09:45.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-03-03T06:09:45.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-03-03T06:09:45.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-03-03T06:09:45.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-03-03T06:09:45.857 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-03-03T06:09:45.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-03-03T06:09:45.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-03-03T06:09:45.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-03-03T06:09:45.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 79953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebtables.ko 2023-03-03T06:09:45.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-03-03T06:09:45.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-03-03T06:09:45.858 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ceph/ 2023-03-03T06:09:45.858 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1047633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ceph/libceph.ko 2023-03-03T06:09:45.859 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/core/ 2023-03-03T06:09:45.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 129729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/core/pktgen.ko 2023-03-03T06:09:45.859 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dccp/ 2023-03-03T06:09:45.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 192145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dccp/dccp.ko 2023-03-03T06:09:45.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dccp/dccp_diag.ko 2023-03-03T06:09:45.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 59065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dccp/dccp_ipv4.ko 2023-03-03T06:09:45.859 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dccp/dccp_ipv6.ko 2023-03-03T06:09:45.859 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dsa/ 2023-03-03T06:09:45.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 276553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dsa/dsa_core.ko 2023-03-03T06:09:45.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dsa/tag_brcm.ko 2023-03-03T06:09:45.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dsa/tag_dsa.ko 2023-03-03T06:09:45.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dsa/tag_none.ko 2023-03-03T06:09:45.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dsa/tag_trailer.ko 2023-03-03T06:09:45.860 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/hsr/ 2023-03-03T06:09:45.860 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 99145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/hsr/hsr.ko 2023-03-03T06:09:45.861 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ieee802154/ 2023-03-03T06:09:45.861 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ieee802154/6lowpan/ 2023-03-03T06:09:45.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 45217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-03-03T06:09:45.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 214049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ieee802154/ieee802154.ko 2023-03-03T06:09:45.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ieee802154/ieee802154_socket.ko 2023-03-03T06:09:45.861 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ 2023-03-03T06:09:45.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ah4.ko 2023-03-03T06:09:45.861 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/esp4.ko 2023-03-03T06:09:45.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 49169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/fou.ko 2023-03-03T06:09:45.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/gre.ko 2023-03-03T06:09:45.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/inet_diag.ko 2023-03-03T06:09:45.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 65561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ip_gre.ko 2023-03-03T06:09:45.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 65033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ip_tunnel.ko 2023-03-03T06:09:45.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ip_vti.ko 2023-03-03T06:09:45.862 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ipcomp.ko 2023-03-03T06:09:45.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ipip.ko 2023-03-03T06:09:45.863 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ 2023-03-03T06:09:45.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-03-03T06:09:45.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-03-03T06:09:45.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-03-03T06:09:45.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 54849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ip_tables.ko 2023-03-03T06:09:45.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 48777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ipt_CLUSTERIP.ko 2023-03-03T06:09:45.863 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-03-03T06:09:45.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-03-03T06:09:45.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-03-03T06:09:45.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-03-03T06:09:45.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-03-03T06:09:45.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-03-03T06:09:45.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-03-03T06:09:45.864 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-03-03T06:09:45.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-03-03T06:09:45.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-03-03T06:09:45.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-03-03T06:09:45.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-03-03T06:09:45.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-03-03T06:09:45.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-03-03T06:09:45.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-03-03T06:09:45.865 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-03-03T06:09:45.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-03-03T06:09:45.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-03-03T06:09:45.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-03-03T06:09:45.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-03-03T06:09:45.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-03-03T06:09:45.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_bic.ko 2023-03-03T06:09:45.866 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_dctcp.ko 2023-03-03T06:09:45.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_diag.ko 2023-03-03T06:09:45.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_highspeed.ko 2023-03-03T06:09:45.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_htcp.ko 2023-03-03T06:09:45.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_hybla.ko 2023-03-03T06:09:45.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_illinois.ko 2023-03-03T06:09:45.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_lp.ko 2023-03-03T06:09:45.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_scalable.ko 2023-03-03T06:09:45.867 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_vegas.ko 2023-03-03T06:09:45.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_veno.ko 2023-03-03T06:09:45.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_westwood.ko 2023-03-03T06:09:45.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_yeah.ko 2023-03-03T06:09:45.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tunnel4.ko 2023-03-03T06:09:45.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/udp_diag.ko 2023-03-03T06:09:45.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/udp_tunnel.ko 2023-03-03T06:09:45.868 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-03-03T06:09:45.868 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ 2023-03-03T06:09:45.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ah6.ko 2023-03-03T06:09:45.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/esp6.ko 2023-03-03T06:09:45.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/fou6.ko 2023-03-03T06:09:45.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 96425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ip6_gre.ko 2023-03-03T06:09:45.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 102673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ip6_tunnel.ko 2023-03-03T06:09:45.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-03-03T06:09:45.869 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ip6_vti.ko 2023-03-03T06:09:45.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ipcomp6.ko 2023-03-03T06:09:45.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/mip6.ko 2023-03-03T06:09:45.870 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ 2023-03-03T06:09:45.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 55449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-03-03T06:09:45.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-03-03T06:09:45.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-03-03T06:09:45.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-03-03T06:09:45.870 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-03-03T06:09:45.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-03-03T06:09:45.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-03-03T06:09:45.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-03-03T06:09:45.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-03-03T06:09:45.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-03-03T06:09:45.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-03-03T06:09:45.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-03-03T06:09:45.871 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-03-03T06:09:45.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-03-03T06:09:45.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-03-03T06:09:45.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-03-03T06:09:45.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-03-03T06:09:45.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-03-03T06:09:45.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-03-03T06:09:45.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-03-03T06:09:45.872 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-03-03T06:09:45.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-03-03T06:09:45.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-03-03T06:09:45.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-03-03T06:09:45.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-03-03T06:09:45.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 90585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/sit.ko 2023-03-03T06:09:45.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/tunnel6.ko 2023-03-03T06:09:45.873 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-03-03T06:09:45.874 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/key/ 2023-03-03T06:09:45.874 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 92481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/key/af_key.ko 2023-03-03T06:09:45.874 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/ 2023-03-03T06:09:45.874 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 133505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_core.ko 2023-03-03T06:09:45.874 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_debugfs.ko 2023-03-03T06:09:45.874 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_eth.ko 2023-03-03T06:09:45.874 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_ip.ko 2023-03-03T06:09:45.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_ip6.ko 2023-03-03T06:09:45.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_netlink.ko 2023-03-03T06:09:45.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_ppp.ko 2023-03-03T06:09:45.875 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/lapb/ 2023-03-03T06:09:45.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 46593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/lapb/lapb.ko 2023-03-03T06:09:45.875 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/llc/ 2023-03-03T06:09:45.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/llc/llc.ko 2023-03-03T06:09:45.875 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 266329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/llc/llc2.ko 2023-03-03T06:09:45.876 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/mac802154/ 2023-03-03T06:09:45.876 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 238257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/mac802154/mac802154.ko 2023-03-03T06:09:45.876 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/mpls/ 2023-03-03T06:09:45.876 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/mpls/mpls_gso.ko 2023-03-03T06:09:45.876 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ 2023-03-03T06:09:45.876 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ 2023-03-03T06:09:45.876 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 135785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set.ko 2023-03-03T06:09:45.876 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-03-03T06:09:45.877 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-03-03T06:09:45.877 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-03-03T06:09:45.877 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 101393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-03-03T06:09:45.877 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 102569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-03-03T06:09:45.877 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 104065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-03-03T06:09:45.877 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 104529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-03-03T06:09:45.877 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 112081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-03-03T06:09:45.878 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 60705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-03-03T06:09:45.878 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 108729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-03-03T06:09:45.878 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 112361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-03-03T06:09:45.878 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 111865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-03-03T06:09:45.878 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 110785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-03-03T06:09:45.878 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 113737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-03-03T06:09:45.878 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-03-03T06:09:45.878 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ 2023-03-03T06:09:45.879 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 407209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-03-03T06:09:45.879 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-03-03T06:09:45.879 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-03-03T06:09:45.879 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-03-03T06:09:45.879 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 17409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-03-03T06:09:45.879 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-03-03T06:09:45.879 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-03-03T06:09:45.880 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-03-03T06:09:45.880 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-03-03T06:09:45.880 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-03-03T06:09:45.880 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-03-03T06:09:45.880 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-03-03T06:09:45.880 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-03-03T06:09:45.880 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-03-03T06:09:45.880 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conncount.ko 2023-03-03T06:09:45.881 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 429161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack.ko 2023-03-03T06:09:45.881 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-03-03T06:09:45.881 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-03-03T06:09:45.881 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-03-03T06:09:45.881 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 116537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-03-03T06:09:45.881 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-03-03T06:09:45.881 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-03-03T06:09:45.881 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 105961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-03-03T06:09:45.882 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-03-03T06:09:45.882 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-03-03T06:09:45.882 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 64865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-03-03T06:09:45.882 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-03-03T06:09:45.882 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-03-03T06:09:45.882 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_dup_netdev.ko 2023-03-03T06:09:45.882 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 85633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_flow_table.ko 2023-03-03T06:09:45.883 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-03-03T06:09:45.883 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 36577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_log_syslog.ko 2023-03-03T06:09:45.883 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 91809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_nat.ko 2023-03-03T06:09:45.883 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_nat_amanda.ko 2023-03-03T06:09:45.883 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_nat_ftp.ko 2023-03-03T06:09:45.883 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_nat_irc.ko 2023-03-03T06:09:45.883 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_nat_sip.ko 2023-03-03T06:09:45.883 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_nat_tftp.ko 2023-03-03T06:09:45.884 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 51305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_synproxy_core.ko 2023-03-03T06:09:45.884 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 644793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_tables.ko 2023-03-03T06:09:45.884 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nfnetlink.ko 2023-03-03T06:09:45.884 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 34825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nfnetlink_acct.ko 2023-03-03T06:09:45.884 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-03-03T06:09:45.884 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 47201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nfnetlink_log.ko 2023-03-03T06:09:45.884 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nfnetlink_osf.ko 2023-03-03T06:09:45.885 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 61729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nfnetlink_queue.ko 2023-03-03T06:09:45.885 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_chain_nat.ko 2023-03-03T06:09:45.885 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_compat.ko 2023-03-03T06:09:45.885 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 40121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_ct.ko 2023-03-03T06:09:45.885 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_dup_netdev.ko 2023-03-03T06:09:45.885 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_fib.ko 2023-03-03T06:09:45.886 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-03-03T06:09:45.886 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_hash.ko 2023-03-03T06:09:45.886 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_limit.ko 2023-03-03T06:09:45.886 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_log.ko 2023-03-03T06:09:45.886 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_masq.ko 2023-03-03T06:09:45.886 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_nat.ko 2023-03-03T06:09:45.886 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_numgen.ko 2023-03-03T06:09:45.886 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_osf.ko 2023-03-03T06:09:45.887 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_queue.ko 2023-03-03T06:09:45.887 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_redir.ko 2023-03-03T06:09:45.887 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_reject.ko 2023-03-03T06:09:45.887 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_reject_inet.ko 2023-03-03T06:09:45.887 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_socket.ko 2023-03-03T06:09:45.887 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_synproxy.ko 2023-03-03T06:09:45.887 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_tproxy.ko 2023-03-03T06:09:45.887 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 105865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/x_tables.ko 2023-03-03T06:09:45.888 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_AUDIT.ko 2023-03-03T06:09:45.888 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-03-03T06:09:45.888 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-03-03T06:09:45.888 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-03-03T06:09:45.888 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_CT.ko 2023-03-03T06:09:45.888 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_DSCP.ko 2023-03-03T06:09:45.888 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_HL.ko 2023-03-03T06:09:45.888 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_HMARK.ko 2023-03-03T06:09:45.889 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 38377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-03-03T06:09:45.889 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_LED.ko 2023-03-03T06:09:45.889 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_LOG.ko 2023-03-03T06:09:45.889 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-03-03T06:09:45.889 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_NETMAP.ko 2023-03-03T06:09:45.889 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_NFLOG.ko 2023-03-03T06:09:45.889 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-03-03T06:09:45.890 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_RATEEST.ko 2023-03-03T06:09:45.890 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_REDIRECT.ko 2023-03-03T06:09:45.890 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_SECMARK.ko 2023-03-03T06:09:45.890 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_TCPMSS.ko 2023-03-03T06:09:45.890 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-03-03T06:09:45.890 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_TEE.ko 2023-03-03T06:09:45.890 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_TPROXY.ko 2023-03-03T06:09:45.890 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_TRACE.ko 2023-03-03T06:09:45.891 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_addrtype.ko 2023-03-03T06:09:45.891 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_bpf.ko 2023-03-03T06:09:45.891 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_cgroup.ko 2023-03-03T06:09:45.891 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_cluster.ko 2023-03-03T06:09:45.891 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7073 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_comment.ko 2023-03-03T06:09:45.891 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_connbytes.ko 2023-03-03T06:09:45.891 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_connlabel.ko 2023-03-03T06:09:45.892 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_connlimit.ko 2023-03-03T06:09:45.892 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_connmark.ko 2023-03-03T06:09:45.892 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_conntrack.ko 2023-03-03T06:09:45.892 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_cpu.ko 2023-03-03T06:09:45.892 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_dccp.ko 2023-03-03T06:09:45.892 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_devgroup.ko 2023-03-03T06:09:45.892 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_dscp.ko 2023-03-03T06:09:45.892 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_ecn.ko 2023-03-03T06:09:45.893 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_esp.ko 2023-03-03T06:09:45.893 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 50401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_hashlimit.ko 2023-03-03T06:09:45.893 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_helper.ko 2023-03-03T06:09:45.893 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_hl.ko 2023-03-03T06:09:45.893 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_ipcomp.ko 2023-03-03T06:09:45.893 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_iprange.ko 2023-03-03T06:09:45.893 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_ipvs.ko 2023-03-03T06:09:45.894 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_l2tp.ko 2023-03-03T06:09:45.894 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_length.ko 2023-03-03T06:09:45.894 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_limit.ko 2023-03-03T06:09:45.894 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_mac.ko 2023-03-03T06:09:45.894 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_mark.ko 2023-03-03T06:09:45.894 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_multiport.ko 2023-03-03T06:09:45.894 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_nat.ko 2023-03-03T06:09:45.894 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_nfacct.ko 2023-03-03T06:09:45.895 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_osf.ko 2023-03-03T06:09:45.895 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_owner.ko 2023-03-03T06:09:45.895 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_physdev.ko 2023-03-03T06:09:45.895 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_pkttype.ko 2023-03-03T06:09:45.895 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_policy.ko 2023-03-03T06:09:45.895 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_quota.ko 2023-03-03T06:09:45.895 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_rateest.ko 2023-03-03T06:09:45.895 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_realm.ko 2023-03-03T06:09:45.896 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 41633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_recent.ko 2023-03-03T06:09:45.896 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_sctp.ko 2023-03-03T06:09:45.896 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 31201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_set.ko 2023-03-03T06:09:45.896 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 16089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_socket.ko 2023-03-03T06:09:45.896 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_state.ko 2023-03-03T06:09:45.896 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_statistic.ko 2023-03-03T06:09:45.896 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_string.ko 2023-03-03T06:09:45.896 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_tcpmss.ko 2023-03-03T06:09:45.897 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_tcpudp.ko 2023-03-03T06:09:45.897 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_time.ko 2023-03-03T06:09:45.897 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_u32.ko 2023-03-03T06:09:45.897 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netlink/ 2023-03-03T06:09:45.897 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 13417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netlink/netlink_diag.ko 2023-03-03T06:09:45.897 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/nsh/ 2023-03-03T06:09:45.897 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/nsh/nsh.ko 2023-03-03T06:09:45.898 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/openvswitch/ 2023-03-03T06:09:45.898 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 400273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/openvswitch/openvswitch.ko 2023-03-03T06:09:45.898 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/openvswitch/vport-geneve.ko 2023-03-03T06:09:45.898 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/openvswitch/vport-gre.ko 2023-03-03T06:09:45.898 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 11297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/openvswitch/vport-vxlan.ko 2023-03-03T06:09:45.898 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/packet/ 2023-03-03T06:09:45.898 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/packet/af_packet_diag.ko 2023-03-03T06:09:45.898 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/phonet/ 2023-03-03T06:09:45.899 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 124025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/phonet/phonet.ko 2023-03-03T06:09:45.899 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 52937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/phonet/pn_pep.ko 2023-03-03T06:09:45.899 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/rds/ 2023-03-03T06:09:45.899 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 251929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/rds/rds.ko 2023-03-03T06:09:45.899 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 172945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/rds/rds_rdma.ko 2023-03-03T06:09:45.899 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 70913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/rds/rds_tcp.ko 2023-03-03T06:09:45.899 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/rxrpc/ 2023-03-03T06:09:45.899 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 781177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/rxrpc/rxrpc.ko 2023-03-03T06:09:45.900 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/ 2023-03-03T06:09:45.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_csum.ko 2023-03-03T06:09:45.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 25305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_gact.ko 2023-03-03T06:09:45.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 29753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_ipt.ko 2023-03-03T06:09:45.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_mirred.ko 2023-03-03T06:09:45.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_nat.ko 2023-03-03T06:09:45.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_pedit.ko 2023-03-03T06:09:45.900 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_police.ko 2023-03-03T06:09:45.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 21681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_simple.ko 2023-03-03T06:09:45.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_skbedit.ko 2023-03-03T06:09:45.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 37825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_vlan.ko 2023-03-03T06:09:45.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_basic.ko 2023-03-03T06:09:45.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_bpf.ko 2023-03-03T06:09:45.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_cgroup.ko 2023-03-03T06:09:45.901 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_flow.ko 2023-03-03T06:09:45.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_fw.ko 2023-03-03T06:09:45.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_route.ko 2023-03-03T06:09:45.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_rsvp.ko 2023-03-03T06:09:45.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 35033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_rsvp6.ko 2023-03-03T06:09:45.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 42041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_tcindex.ko 2023-03-03T06:09:45.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 56441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_u32.ko 2023-03-03T06:09:45.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/em_cmp.ko 2023-03-03T06:09:45.902 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 12857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/em_ipset.ko 2023-03-03T06:09:45.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/em_meta.ko 2023-03-03T06:09:45.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 7393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/em_nbyte.ko 2023-03-03T06:09:45.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/em_text.ko 2023-03-03T06:09:45.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 6753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/em_u32.ko 2023-03-03T06:09:45.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 43737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_cbq.ko 2023-03-03T06:09:45.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_choke.ko 2023-03-03T06:09:45.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 19201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_codel.ko 2023-03-03T06:09:45.903 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_drr.ko 2023-03-03T06:09:45.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_dsmark.ko 2023-03-03T06:09:45.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 30153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_fq.ko 2023-03-03T06:09:45.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_fq_codel.ko 2023-03-03T06:09:45.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 32729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_gred.ko 2023-03-03T06:09:45.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_hfsc.ko 2023-03-03T06:09:45.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_hhf.ko 2023-03-03T06:09:45.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_htb.ko 2023-03-03T06:09:45.904 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_ingress.ko 2023-03-03T06:09:45.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_mqprio.ko 2023-03-03T06:09:45.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_multiq.ko 2023-03-03T06:09:45.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 33649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_netem.ko 2023-03-03T06:09:45.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 23497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_pie.ko 2023-03-03T06:09:45.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_plug.ko 2023-03-03T06:09:45.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_prio.ko 2023-03-03T06:09:45.905 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 44409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_qfq.ko 2023-03-03T06:09:45.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 28609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_red.ko 2023-03-03T06:09:45.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 24769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_sfb.ko 2023-03-03T06:09:45.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_sfq.ko 2023-03-03T06:09:45.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 27209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_tbf.ko 2023-03-03T06:09:45.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 26641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_teql.ko 2023-03-03T06:09:45.906 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sctp/ 2023-03-03T06:09:45.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 921329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sctp/sctp.ko 2023-03-03T06:09:45.906 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sctp/sctp_diag.ko 2023-03-03T06:09:45.907 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/ 2023-03-03T06:09:45.907 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/auth_gss/ 2023-03-03T06:09:45.907 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 339241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-03-03T06:09:45.907 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 63193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-03-03T06:09:45.907 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1476289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/sunrpc.ko 2023-03-03T06:09:45.907 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/xprtrdma/ 2023-03-03T06:09:45.907 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 720761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-03-03T06:09:45.908 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/tipc/ 2023-03-03T06:09:45.908 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 9529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/tipc/diag.ko 2023-03-03T06:09:45.908 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 1123321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/tipc/tipc.ko 2023-03-03T06:09:45.908 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/unix/ 2023-03-03T06:09:45.908 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 15233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/unix/unix_diag.ko 2023-03-03T06:09:45.908 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/vmw_vsock/ 2023-03-03T06:09:45.908 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 77785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-03-03T06:09:45.908 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 68289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-03-03T06:09:45.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 82433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/vmw_vsock/vsock.ko 2023-03-03T06:09:45.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/vmw_vsock/vsock_diag.ko 2023-03-03T06:09:45.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 18393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-03-03T06:09:45.909 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/x25/ 2023-03-03T06:09:45.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 125817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/x25/x25.ko 2023-03-03T06:09:45.909 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/xfrm/ 2023-03-03T06:09:45.909 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/xfrm/xfrm_algo.ko 2023-03-03T06:09:45.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 20009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-03-03T06:09:45.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 101665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/xfrm/xfrm_user.ko 2023-03-03T06:09:45.910 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/virt/ 2023-03-03T06:09:45.910 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/virt/lib/ 2023-03-03T06:09:45.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 14433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/virt/lib/irqbypass.ko 2023-03-03T06:09:45.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 633131 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.alias 2023-03-03T06:09:45.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 619504 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.alias.bin 2023-03-03T06:09:45.910 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 8377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.builtin 2023-03-03T06:09:45.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 22490 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.builtin.alias.bin 2023-03-03T06:09:45.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 10648 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.builtin.bin 2023-03-03T06:09:45.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 69838 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.builtin.modinfo 2023-03-03T06:09:45.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 147260 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.dep 2023-03-03T06:09:45.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 234530 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.dep.bin 2023-03-03T06:09:45.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 210 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.devname 2023-03-03T06:09:45.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 73341 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.order 2023-03-03T06:09:45.911 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 830 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.softdep 2023-03-03T06:09:45.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 250429 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.symbols 2023-03-03T06:09:45.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 304835 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.symbols.bin 2023-03-03T06:09:45.912 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./usr/ 2023-03-03T06:09:45.912 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./usr/share/ 2023-03-03T06:09:45.912 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./usr/share/doc/ 2023-03-03T06:09:45.912 INFO:teuthology.orchestra.run.smithi003.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./usr/share/doc/linux-image-6.2.0-ceph-g75534f74ac3d/ 2023-03-03T06:09:45.912 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 176 2023-03-03 04:49 ./usr/share/doc/linux-image-6.2.0-ceph-g75534f74ac3d/changelog.Debian.gz 2023-03-03T06:09:45.913 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r-- root/root 692 2023-03-03 04:49 ./usr/share/doc/linux-image-6.2.0-ceph-g75534f74ac3d/copyright 2023-03-03T06:09:45.914 DEBUG:teuthology.task.kernel:get_image_version: 6.2.0-ceph-g75534f74ac3d 2023-03-03T06:09:45.914 INFO:teuthology.task.kernel:searching for kernel 6.2.0-ceph-g75534f74ac3d 2023-03-03T06:09:45.914 DEBUG:teuthology.orchestra.run.smithi003:> egrep '(submenu|menuentry.*6.2.0-ceph-g75534f74ac3d).*{' /boot/grub/grub.cfg 2023-03-03T06:09:45.928 INFO:teuthology.orchestra.run.smithi003.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-03-03T06:09:45.928 INFO:teuthology.orchestra.run.smithi003.stdout: menuentry 'Ubuntu, with Linux 6.2.0-ceph-g75534f74ac3d' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.2.0-ceph-g75534f74ac3d-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-03-03T06:09:45.928 INFO:teuthology.orchestra.run.smithi003.stdout: menuentry 'Ubuntu, with Linux 6.2.0-ceph-g75534f74ac3d (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.2.0-ceph-g75534f74ac3d-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-03-03T06:09:45.929 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-03-03T06:09:45.929 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.2.0-ceph-g75534f74ac3d 2023-03-03T06:09:45.929 DEBUG:teuthology.orchestra.run.smithi003:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-03-03T06:09:45.932 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi003.front.sepia.ceph.com to complete... 2023-03-03T06:09:46.654 INFO:teuthology.orchestra.run.smithi003.stderr:Sourcing file `/etc/default/grub' 2023-03-03T06:09:46.662 INFO:teuthology.orchestra.run.smithi003.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-03-03T06:09:46.675 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2023-03-03T06:09:46.899 INFO:teuthology.orchestra.run.smithi003.stderr:Found linux image: /boot/vmlinuz-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:09:46.910 INFO:teuthology.orchestra.run.smithi003.stderr:Found initrd image: /boot/initrd.img-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:09:47.246 INFO:teuthology.orchestra.run.smithi003.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-03-03T06:09:47.258 INFO:teuthology.orchestra.run.smithi003.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-03-03T06:09:48.347 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-6.2.0-ceph-g75534f74ac3d (6.2.0-ceph-g75534f74ac3d-1) ... 2023-03-03T06:09:48.494 INFO:teuthology.orchestra.run.smithi032.stderr:update-initramfs: Generating /boot/initrd.img-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:09:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:done 2023-03-03T06:10:04.567 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub' 2023-03-03T06:10:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-03-03T06:10:04.602 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2023-03-03T06:10:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:10:04.941 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:10:05.365 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-03-03T06:10:05.377 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-03-03T06:10:08.006 INFO:teuthology.orchestra.run.smithi032.stderr:done 2023-03-03T06:10:08.227 DEBUG:teuthology.orchestra.run.smithi032:> dpkg-deb -c /tmp/linux-image.deb 2023-03-03T06:10:08.237 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./ 2023-03-03T06:10:08.237 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./boot/ 2023-03-03T06:10:08.237 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 4431288 2023-03-03 04:49 ./boot/System.map-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:10:08.253 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 166345 2023-03-03 04:49 ./boot/config-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:10:08.254 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11098304 2023-03-03 04:49 ./boot/vmlinuz-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:10:08.274 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./etc/ 2023-03-03T06:10:08.275 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./etc/kernel/ 2023-03-03T06:10:08.275 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./etc/kernel/postinst.d/ 2023-03-03T06:10:08.275 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./etc/kernel/postrm.d/ 2023-03-03T06:10:08.275 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./etc/kernel/preinst.d/ 2023-03-03T06:10:08.275 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./etc/kernel/prerm.d/ 2023-03-03T06:10:08.275 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/ 2023-03-03T06:10:08.276 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/ 2023-03-03T06:10:08.276 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/ 2023-03-03T06:10:08.276 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/ 2023-03-03T06:10:08.276 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/ 2023-03-03T06:10:08.276 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/ 2023-03-03T06:10:08.276 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/ 2023-03-03T06:10:08.276 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 527801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/aesni-intel.ko 2023-03-03T06:10:08.278 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-03-03T06:10:08.279 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-03-03T06:10:08.279 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-03-03T06:10:08.279 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 88369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-03-03T06:10:08.279 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-03-03T06:10:08.279 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 142161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-03-03T06:10:08.279 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-03-03T06:10:08.280 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-03-03T06:10:08.280 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-03-03T06:10:08.280 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-03-03T06:10:08.280 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-03-03T06:10:08.280 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-03-03T06:10:08.280 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-03-03T06:10:08.280 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-03-03T06:10:08.280 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-03-03T06:10:08.281 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-03-03T06:10:08.281 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-03-03T06:10:08.281 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-03-03T06:10:08.281 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-03-03T06:10:08.281 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kernel/ 2023-03-03T06:10:08.281 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kernel/cpu/ 2023-03-03T06:10:08.281 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kernel/cpu/mce/ 2023-03-03T06:10:08.282 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-03-03T06:10:08.282 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kernel/cpuid.ko 2023-03-03T06:10:08.282 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kernel/msr.ko 2023-03-03T06:10:08.282 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kvm/ 2023-03-03T06:10:08.282 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 344241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kvm/kvm-amd.ko 2023-03-03T06:10:08.282 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 912137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kvm/kvm-intel.ko 2023-03-03T06:10:08.283 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2209417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/arch/x86/kvm/kvm.ko 2023-03-03T06:10:08.291 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/ 2023-03-03T06:10:08.291 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/af_alg.ko 2023-03-03T06:10:08.291 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/algif_hash.ko 2023-03-03T06:10:08.291 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/algif_skcipher.ko 2023-03-03T06:10:08.291 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/ansi_cprng.ko 2023-03-03T06:10:08.291 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/anubis.ko 2023-03-03T06:10:08.292 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/arc4.ko 2023-03-03T06:10:08.292 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/asymmetric_keys/ 2023-03-03T06:10:08.292 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-03-03T06:10:08.292 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/ 2023-03-03T06:10:08.292 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/async_memcpy.ko 2023-03-03T06:10:08.292 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/async_pq.ko 2023-03-03T06:10:08.292 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-03-03T06:10:08.292 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/async_tx.ko 2023-03-03T06:10:08.293 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/async_xor.ko 2023-03-03T06:10:08.293 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/async_tx/raid6test.ko 2023-03-03T06:10:08.293 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/authenc.ko 2023-03-03T06:10:08.293 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/authencesn.ko 2023-03-03T06:10:08.293 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/blake2b_generic.ko 2023-03-03T06:10:08.293 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/blowfish_common.ko 2023-03-03T06:10:08.294 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/blowfish_generic.ko 2023-03-03T06:10:08.294 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/camellia_generic.ko 2023-03-03T06:10:08.294 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/cast5_generic.ko 2023-03-03T06:10:08.294 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/cast6_generic.ko 2023-03-03T06:10:08.294 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/cast_common.ko 2023-03-03T06:10:08.294 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/ccm.ko 2023-03-03T06:10:08.294 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/cmac.ko 2023-03-03T06:10:08.295 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/crc32_generic.ko 2023-03-03T06:10:08.295 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/cryptd.ko 2023-03-03T06:10:08.295 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/crypto_engine.ko 2023-03-03T06:10:08.295 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/crypto_simd.ko 2023-03-03T06:10:08.295 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/crypto_user.ko 2023-03-03T06:10:08.295 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/cts.ko 2023-03-03T06:10:08.295 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/des_generic.ko 2023-03-03T06:10:08.296 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/dh_generic.ko 2023-03-03T06:10:08.296 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/echainiv.ko 2023-03-03T06:10:08.296 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/essiv.ko 2023-03-03T06:10:08.296 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/fcrypt.ko 2023-03-03T06:10:08.296 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/khazad.ko 2023-03-03T06:10:08.296 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/lrw.ko 2023-03-03T06:10:08.296 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/lz4.ko 2023-03-03T06:10:08.296 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/lz4hc.ko 2023-03-03T06:10:08.297 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/md4.ko 2023-03-03T06:10:08.297 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/michael_mic.ko 2023-03-03T06:10:08.297 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/pcbc.ko 2023-03-03T06:10:08.297 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/pcrypt.ko 2023-03-03T06:10:08.297 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/rmd160.ko 2023-03-03T06:10:08.297 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/seed.ko 2023-03-03T06:10:08.297 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/serpent_generic.ko 2023-03-03T06:10:08.298 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 145489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/tcrypt.ko 2023-03-03T06:10:08.298 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/tea.ko 2023-03-03T06:10:08.298 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/twofish_common.ko 2023-03-03T06:10:08.298 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/twofish_generic.ko 2023-03-03T06:10:08.298 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/vmac.ko 2023-03-03T06:10:08.298 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/wp512.ko 2023-03-03T06:10:08.298 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/xcbc.ko 2023-03-03T06:10:08.298 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/xor.ko 2023-03-03T06:10:08.299 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/xts.ko 2023-03-03T06:10:08.299 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/crypto/xxhash_generic.ko 2023-03-03T06:10:08.299 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ 2023-03-03T06:10:08.299 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/ 2023-03-03T06:10:08.299 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/acpi_extlog.ko 2023-03-03T06:10:08.299 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/acpi_ipmi.ko 2023-03-03T06:10:08.300 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/acpi_pad.ko 2023-03-03T06:10:08.300 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/apei/ 2023-03-03T06:10:08.301 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/apei/einj.ko 2023-03-03T06:10:08.301 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/ec_sys.ko 2023-03-03T06:10:08.301 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/platform_profile.ko 2023-03-03T06:10:08.301 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/sbs.ko 2023-03-03T06:10:08.301 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/sbshc.ko 2023-03-03T06:10:08.301 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/acpi/video.ko 2023-03-03T06:10:08.301 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/ 2023-03-03T06:10:08.302 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/acard-ahci.ko 2023-03-03T06:10:08.302 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/ahci.ko 2023-03-03T06:10:08.302 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/ahci_platform.ko 2023-03-03T06:10:08.302 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 86265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/libahci.ko 2023-03-03T06:10:08.302 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/libahci_platform.ko 2023-03-03T06:10:08.303 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_acpi.ko 2023-03-03T06:10:08.303 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_ali.ko 2023-03-03T06:10:08.303 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_amd.ko 2023-03-03T06:10:08.303 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_artop.ko 2023-03-03T06:10:08.303 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_atiixp.ko 2023-03-03T06:10:08.303 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_atp867x.ko 2023-03-03T06:10:08.303 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_cmd640.ko 2023-03-03T06:10:08.304 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_cmd64x.ko 2023-03-03T06:10:08.304 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_cypress.ko 2023-03-03T06:10:08.304 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_efar.ko 2023-03-03T06:10:08.304 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_hpt366.ko 2023-03-03T06:10:08.304 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_hpt37x.ko 2023-03-03T06:10:08.304 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23073 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-03-03T06:10:08.304 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_hpt3x3.ko 2023-03-03T06:10:08.304 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_it8213.ko 2023-03-03T06:10:08.305 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_it821x.ko 2023-03-03T06:10:08.305 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_jmicron.ko 2023-03-03T06:10:08.305 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_legacy.ko 2023-03-03T06:10:08.305 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_marvell.ko 2023-03-03T06:10:08.305 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_mpiix.ko 2023-03-03T06:10:08.305 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_netcell.ko 2023-03-03T06:10:08.305 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_ninja32.ko 2023-03-03T06:10:08.306 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_ns87410.ko 2023-03-03T06:10:08.306 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_ns87415.ko 2023-03-03T06:10:08.306 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_oldpiix.ko 2023-03-03T06:10:08.306 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_opti.ko 2023-03-03T06:10:08.306 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_optidma.ko 2023-03-03T06:10:08.306 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_pcmcia.ko 2023-03-03T06:10:08.306 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_pdc2027x.ko 2023-03-03T06:10:08.307 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-03-03T06:10:08.307 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_piccolo.ko 2023-03-03T06:10:08.307 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_radisys.ko 2023-03-03T06:10:08.307 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_rdc.ko 2023-03-03T06:10:08.307 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_rz1000.ko 2023-03-03T06:10:08.307 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_sch.ko 2023-03-03T06:10:08.307 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_serverworks.ko 2023-03-03T06:10:08.307 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_sil680.ko 2023-03-03T06:10:08.308 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_sl82c105.ko 2023-03-03T06:10:08.308 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_triflex.ko 2023-03-03T06:10:08.308 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pata_via.ko 2023-03-03T06:10:08.308 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/pdc_adma.ko 2023-03-03T06:10:08.308 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_inic162x.ko 2023-03-03T06:10:08.308 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_mv.ko 2023-03-03T06:10:08.308 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_nv.ko 2023-03-03T06:10:08.308 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_promise.ko 2023-03-03T06:10:08.309 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_qstor.ko 2023-03-03T06:10:08.309 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_sil.ko 2023-03-03T06:10:08.309 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_sil24.ko 2023-03-03T06:10:08.309 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_sis.ko 2023-03-03T06:10:08.309 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_svw.ko 2023-03-03T06:10:08.309 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_sx4.ko 2023-03-03T06:10:08.309 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_uli.ko 2023-03-03T06:10:08.310 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_via.ko 2023-03-03T06:10:08.310 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ata/sata_vsc.ko 2023-03-03T06:10:08.310 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/auxdisplay/ 2023-03-03T06:10:08.310 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/auxdisplay/ks0108.ko 2023-03-03T06:10:08.310 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/bcma/ 2023-03-03T06:10:08.310 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 123145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/bcma/bcma.ko 2023-03-03T06:10:08.310 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/ 2023-03-03T06:10:08.310 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/aoe/ 2023-03-03T06:10:08.311 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 115577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/aoe/aoe.ko 2023-03-03T06:10:08.311 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/drbd/ 2023-03-03T06:10:08.311 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 980033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/drbd/drbd.ko 2023-03-03T06:10:08.347 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 175665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/floppy.ko 2023-03-03T06:10:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/mtip32xx/ 2023-03-03T06:10:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-03-03T06:10:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 120025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/nbd.ko 2023-03-03T06:10:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/null_blk/ 2023-03-03T06:10:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 111145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/null_blk/null_blk.ko 2023-03-03T06:10:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/pktcdvd.ko 2023-03-03T06:10:08.348 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 262345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/rbd.ko 2023-03-03T06:10:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/xen-blkback/ 2023-03-03T06:10:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-03-03T06:10:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/zram/ 2023-03-03T06:10:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/block/zram/zram.ko 2023-03-03T06:10:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ 2023-03-03T06:10:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/applicom.ko 2023-03-03T06:10:08.349 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hangcheck-timer.ko 2023-03-03T06:10:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hw_random/ 2023-03-03T06:10:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hw_random/amd-rng.ko 2023-03-03T06:10:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hw_random/intel-rng.ko 2023-03-03T06:10:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-03-03T06:10:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hw_random/via-rng.ko 2023-03-03T06:10:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-03-03T06:10:08.350 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ 2023-03-03T06:10:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-03-03T06:10:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 150129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-03-03T06:10:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-03-03T06:10:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 142969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-03-03T06:10:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-03-03T06:10:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-03-03T06:10:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/lp.ko 2023-03-03T06:10:08.351 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/mwave/ 2023-03-03T06:10:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/mwave/mwave.ko 2023-03-03T06:10:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/nvram.ko 2023-03-03T06:10:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/pcmcia/ 2023-03-03T06:10:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/pcmcia/cm4000_cs.ko 2023-03-03T06:10:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/pcmcia/cm4040_cs.ko 2023-03-03T06:10:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/pcmcia/synclink_cs.ko 2023-03-03T06:10:08.352 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/ppdev.ko 2023-03-03T06:10:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tlclk.ko 2023-03-03T06:10:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/ 2023-03-03T06:10:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-03-03T06:10:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-03-03T06:10:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-03-03T06:10:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-03-03T06:10:08.353 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-03-03T06:10:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-03-03T06:10:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-03-03T06:10:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/xillybus/ 2023-03-03T06:10:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/xillybus/xillybus_class.ko 2023-03-03T06:10:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/xillybus/xillybus_core.ko 2023-03-03T06:10:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-03-03T06:10:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/clk/ 2023-03-03T06:10:08.354 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/clk/clk-palmas.ko 2023-03-03T06:10:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/clk/clk-si5351.ko 2023-03-03T06:10:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/clk/clk-twl6040.ko 2023-03-03T06:10:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/clk/clk-wm831x.ko 2023-03-03T06:10:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/cpufreq/ 2023-03-03T06:10:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-03-03T06:10:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-03-03T06:10:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-03-03T06:10:08.355 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/ 2023-03-03T06:10:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/ccp/ 2023-03-03T06:10:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-03-03T06:10:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 225865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/ccp/ccp.ko 2023-03-03T06:10:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/padlock-aes.ko 2023-03-03T06:10:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/padlock-sha.ko 2023-03-03T06:10:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/qat/ 2023-03-03T06:10:08.356 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/qat/qat_common/ 2023-03-03T06:10:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 415785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/qat/qat_common/intel_qat.ko 2023-03-03T06:10:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/qat/qat_dh895xcc/ 2023-03-03T06:10:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-03-03T06:10:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/virtio/ 2023-03-03T06:10:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-03-03T06:10:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dca/ 2023-03-03T06:10:08.357 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dca/dca.ko 2023-03-03T06:10:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dma/ 2023-03-03T06:10:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dma/dw/ 2023-03-03T06:10:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dma/dw/dw_dmac.ko 2023-03-03T06:10:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dma/ioat/ 2023-03-03T06:10:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/dma/ioat/ioatdma.ko 2023-03-03T06:10:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/ 2023-03-03T06:10:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/amd64_edac.ko 2023-03-03T06:10:08.358 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/e752x_edac.ko 2023-03-03T06:10:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/edac_mce_amd.ko 2023-03-03T06:10:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i3000_edac.ko 2023-03-03T06:10:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i3200_edac.ko 2023-03-03T06:10:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i5100_edac.ko 2023-03-03T06:10:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i5400_edac.ko 2023-03-03T06:10:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i7300_edac.ko 2023-03-03T06:10:08.359 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i7core_edac.ko 2023-03-03T06:10:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/i82975x_edac.ko 2023-03-03T06:10:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/ie31200_edac.ko 2023-03-03T06:10:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/sb_edac.ko 2023-03-03T06:10:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/edac/x38_edac.ko 2023-03-03T06:10:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/ 2023-03-03T06:10:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-gpio.ko 2023-03-03T06:10:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-max14577.ko 2023-03-03T06:10:08.360 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-max77693.ko 2023-03-03T06:10:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-max8997.ko 2023-03-03T06:10:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-palmas.ko 2023-03-03T06:10:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-rt8973a.ko 2023-03-03T06:10:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/extcon/extcon-sm5502.ko 2023-03-03T06:10:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/firmware/ 2023-03-03T06:10:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/firmware/dmi-sysfs.ko 2023-03-03T06:10:08.361 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/firmware/efi/ 2023-03-03T06:10:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/firmware/efi/efi-pstore.ko 2023-03-03T06:10:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/firmware/iscsi_ibft.ko 2023-03-03T06:10:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/ 2023-03-03T06:10:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-adp5520.ko 2023-03-03T06:10:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-amd8111.ko 2023-03-03T06:10:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-arizona.ko 2023-03-03T06:10:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-da9052.ko 2023-03-03T06:10:08.362 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-da9055.ko 2023-03-03T06:10:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-dln2.ko 2023-03-03T06:10:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-f7188x.ko 2023-03-03T06:10:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-generic.ko 2023-03-03T06:10:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-ich.ko 2023-03-03T06:10:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-03-03T06:10:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-kempld.ko 2023-03-03T06:10:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-lp3943.ko 2023-03-03T06:10:08.363 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-max7300.ko 2023-03-03T06:10:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-max730x.ko 2023-03-03T06:10:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-max732x.ko 2023-03-03T06:10:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-03-03T06:10:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-pca953x.ko 2023-03-03T06:10:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-pcf857x.ko 2023-03-03T06:10:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-rdc321x.ko 2023-03-03T06:10:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-sch.ko 2023-03-03T06:10:08.364 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-sch311x.ko 2023-03-03T06:10:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-tps65912.ko 2023-03-03T06:10:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-twl4030.ko 2023-03-03T06:10:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-twl6040.ko 2023-03-03T06:10:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-viperboard.ko 2023-03-03T06:10:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-vx855.ko 2023-03-03T06:10:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-wm831x.ko 2023-03-03T06:10:08.365 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-wm8350.ko 2023-03-03T06:10:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/gpio/gpio-wm8994.ko 2023-03-03T06:10:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/ 2023-03-03T06:10:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-a4tech.ko 2023-03-03T06:10:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-apple.ko 2023-03-03T06:10:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-appleir.ko 2023-03-03T06:10:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-aureal.ko 2023-03-03T06:10:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-axff.ko 2023-03-03T06:10:08.366 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-belkin.ko 2023-03-03T06:10:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-cherry.ko 2023-03-03T06:10:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-chicony.ko 2023-03-03T06:10:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-cp2112.ko 2023-03-03T06:10:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-cypress.ko 2023-03-03T06:10:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-dr.ko 2023-03-03T06:10:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-elecom.ko 2023-03-03T06:10:08.367 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-elo.ko 2023-03-03T06:10:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-emsff.ko 2023-03-03T06:10:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-ezkey.ko 2023-03-03T06:10:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-gaff.ko 2023-03-03T06:10:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-generic.ko 2023-03-03T06:10:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-gt683r.ko 2023-03-03T06:10:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-gyration.ko 2023-03-03T06:10:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-03-03T06:10:08.368 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-03-03T06:10:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-holtekff.ko 2023-03-03T06:10:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-hyperv.ko 2023-03-03T06:10:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-icade.ko 2023-03-03T06:10:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-kensington.ko 2023-03-03T06:10:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-keytouch.ko 2023-03-03T06:10:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-kye.ko 2023-03-03T06:10:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-lcpower.ko 2023-03-03T06:10:08.369 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-led.ko 2023-03-03T06:10:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-lenovo.ko 2023-03-03T06:10:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-lg-g15.ko 2023-03-03T06:10:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-logitech-dj.ko 2023-03-03T06:10:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-03-03T06:10:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 88385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-logitech.ko 2023-03-03T06:10:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-magicmouse.ko 2023-03-03T06:10:08.370 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-microsoft.ko 2023-03-03T06:10:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-monterey.ko 2023-03-03T06:10:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-multitouch.ko 2023-03-03T06:10:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-ntrig.ko 2023-03-03T06:10:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-ortek.ko 2023-03-03T06:10:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-penmount.ko 2023-03-03T06:10:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-petalynx.ko 2023-03-03T06:10:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-picolcd.ko 2023-03-03T06:10:08.371 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-pl.ko 2023-03-03T06:10:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-plantronics.ko 2023-03-03T06:10:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-primax.ko 2023-03-03T06:10:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-rmi.ko 2023-03-03T06:10:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-03-03T06:10:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-common.ko 2023-03-03T06:10:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-isku.ko 2023-03-03T06:10:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-kone.ko 2023-03-03T06:10:08.372 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-03-03T06:10:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-03-03T06:10:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-03-03T06:10:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-lua.ko 2023-03-03T06:10:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-03-03T06:10:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-03-03T06:10:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat-savu.ko 2023-03-03T06:10:08.373 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-roccat.ko 2023-03-03T06:10:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-saitek.ko 2023-03-03T06:10:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-samsung.ko 2023-03-03T06:10:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-sensor-hub.ko 2023-03-03T06:10:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-sjoy.ko 2023-03-03T06:10:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-sony.ko 2023-03-03T06:10:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-speedlink.ko 2023-03-03T06:10:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-steelseries.ko 2023-03-03T06:10:08.374 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-sunplus.ko 2023-03-03T06:10:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-thrustmaster.ko 2023-03-03T06:10:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-tivo.ko 2023-03-03T06:10:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-tmff.ko 2023-03-03T06:10:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-topseed.ko 2023-03-03T06:10:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-twinhan.ko 2023-03-03T06:10:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-uclogic.ko 2023-03-03T06:10:08.375 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-waltop.ko 2023-03-03T06:10:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 145113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-wiimote.ko 2023-03-03T06:10:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-xinmo.ko 2023-03-03T06:10:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-zpff.ko 2023-03-03T06:10:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid-zydacron.ko 2023-03-03T06:10:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 260569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/hid.ko 2023-03-03T06:10:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/uhid.ko 2023-03-03T06:10:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/usbhid/ 2023-03-03T06:10:08.376 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 126433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/usbhid/usbhid.ko 2023-03-03T06:10:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/usbhid/usbkbd.ko 2023-03-03T06:10:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/usbhid/usbmouse.ko 2023-03-03T06:10:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 218593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hid/wacom.ko 2023-03-03T06:10:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hv/ 2023-03-03T06:10:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hv/hv_balloon.ko 2023-03-03T06:10:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 92873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hv/hv_utils.ko 2023-03-03T06:10:08.377 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 309889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hv/hv_vmbus.ko 2023-03-03T06:10:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ 2023-03-03T06:10:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/abituguru.ko 2023-03-03T06:10:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/abituguru3.ko 2023-03-03T06:10:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-03-03T06:10:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ad7414.ko 2023-03-03T06:10:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ad7418.ko 2023-03-03T06:10:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adc128d818.ko 2023-03-03T06:10:08.378 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adm1025.ko 2023-03-03T06:10:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 98161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adm1026.ko 2023-03-03T06:10:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adm1029.ko 2023-03-03T06:10:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adm1031.ko 2023-03-03T06:10:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adm9240.ko 2023-03-03T06:10:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ads7828.ko 2023-03-03T06:10:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adt7410.ko 2023-03-03T06:10:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adt7411.ko 2023-03-03T06:10:08.379 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adt7462.ko 2023-03-03T06:10:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adt7470.ko 2023-03-03T06:10:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adt7475.ko 2023-03-03T06:10:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/adt7x10.ko 2023-03-03T06:10:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/amc6821.ko 2023-03-03T06:10:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/applesmc.ko 2023-03-03T06:10:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/asb100.ko 2023-03-03T06:10:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/asc7621.ko 2023-03-03T06:10:08.380 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/asus_atk0110.ko 2023-03-03T06:10:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/atxp1.ko 2023-03-03T06:10:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/coretemp.ko 2023-03-03T06:10:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/da9052-hwmon.ko 2023-03-03T06:10:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/da9055-hwmon.ko 2023-03-03T06:10:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-03-03T06:10:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/dme1737.ko 2023-03-03T06:10:08.381 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ds1621.ko 2023-03-03T06:10:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ds620.ko 2023-03-03T06:10:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/emc1403.ko 2023-03-03T06:10:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/emc2103.ko 2023-03-03T06:10:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/emc6w201.ko 2023-03-03T06:10:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/f71805f.ko 2023-03-03T06:10:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 94609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/f71882fg.ko 2023-03-03T06:10:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/f75375s.ko 2023-03-03T06:10:08.382 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/fam15h_power.ko 2023-03-03T06:10:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/fschmd.ko 2023-03-03T06:10:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/g760a.ko 2023-03-03T06:10:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/g762.ko 2023-03-03T06:10:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/gl518sm.ko 2023-03-03T06:10:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/gl520sm.ko 2023-03-03T06:10:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/hih6130.ko 2023-03-03T06:10:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/hwmon-vid.ko 2023-03-03T06:10:08.383 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/i5500_temp.ko 2023-03-03T06:10:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/i5k_amb.ko 2023-03-03T06:10:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ibmaem.ko 2023-03-03T06:10:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ibmpex.ko 2023-03-03T06:10:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ina209.ko 2023-03-03T06:10:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ina2xx.ko 2023-03-03T06:10:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 118873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/it87.ko 2023-03-03T06:10:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/jc42.ko 2023-03-03T06:10:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/k10temp.ko 2023-03-03T06:10:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/k8temp.ko 2023-03-03T06:10:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lineage-pem.ko 2023-03-03T06:10:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm63.ko 2023-03-03T06:10:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm73.ko 2023-03-03T06:10:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm75.ko 2023-03-03T06:10:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm77.ko 2023-03-03T06:10:08.385 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm78.ko 2023-03-03T06:10:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm80.ko 2023-03-03T06:10:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm83.ko 2023-03-03T06:10:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm85.ko 2023-03-03T06:10:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm87.ko 2023-03-03T06:10:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm90.ko 2023-03-03T06:10:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm92.ko 2023-03-03T06:10:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 112985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm93.ko 2023-03-03T06:10:08.386 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm95234.ko 2023-03-03T06:10:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm95241.ko 2023-03-03T06:10:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/lm95245.ko 2023-03-03T06:10:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc2945.ko 2023-03-03T06:10:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc4151.ko 2023-03-03T06:10:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc4215.ko 2023-03-03T06:10:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc4222.ko 2023-03-03T06:10:08.387 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc4245.ko 2023-03-03T06:10:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc4260.ko 2023-03-03T06:10:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/ltc4261.ko 2023-03-03T06:10:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max16065.ko 2023-03-03T06:10:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max1619.ko 2023-03-03T06:10:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max1668.ko 2023-03-03T06:10:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max197.ko 2023-03-03T06:10:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max6639.ko 2023-03-03T06:10:08.388 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max6650.ko 2023-03-03T06:10:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/max6697.ko 2023-03-03T06:10:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/mc13783-adc.ko 2023-03-03T06:10:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/mcp3021.ko 2023-03-03T06:10:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-03-03T06:10:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/nct6683.ko 2023-03-03T06:10:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 114801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/nct6775-core.ko 2023-03-03T06:10:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/nct6775.ko 2023-03-03T06:10:08.389 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/nct7802.ko 2023-03-03T06:10:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pc87360.ko 2023-03-03T06:10:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pc87427.ko 2023-03-03T06:10:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pcf8591.ko 2023-03-03T06:10:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/ 2023-03-03T06:10:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-03-03T06:10:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-03-03T06:10:08.390 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-03-03T06:10:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-03-03T06:10:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-03-03T06:10:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-03-03T06:10:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-03-03T06:10:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-03-03T06:10:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-03-03T06:10:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-03-03T06:10:08.391 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-03-03T06:10:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-03-03T06:10:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/powr1220.ko 2023-03-03T06:10:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/sch5627.ko 2023-03-03T06:10:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/sch5636.ko 2023-03-03T06:10:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/sch56xx-common.ko 2023-03-03T06:10:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/sht15.ko 2023-03-03T06:10:08.392 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/sht21.ko 2023-03-03T06:10:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/shtc1.ko 2023-03-03T06:10:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/sis5595.ko 2023-03-03T06:10:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/smm665.ko 2023-03-03T06:10:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/smsc47b397.ko 2023-03-03T06:10:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/smsc47m1.ko 2023-03-03T06:10:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/smsc47m192.ko 2023-03-03T06:10:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/thmc50.ko 2023-03-03T06:10:08.393 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/tmp102.ko 2023-03-03T06:10:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/tmp103.ko 2023-03-03T06:10:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/tmp401.ko 2023-03-03T06:10:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/tmp421.ko 2023-03-03T06:10:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/via-cputemp.ko 2023-03-03T06:10:08.394 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/via686a.ko 2023-03-03T06:10:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/vt1211.ko 2023-03-03T06:10:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/vt8231.ko 2023-03-03T06:10:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83627ehf.ko 2023-03-03T06:10:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 79145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83627hf.ko 2023-03-03T06:10:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72073 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83781d.ko 2023-03-03T06:10:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83791d.ko 2023-03-03T06:10:08.395 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83792d.ko 2023-03-03T06:10:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83793.ko 2023-03-03T06:10:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 119897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83795.ko 2023-03-03T06:10:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83l785ts.ko 2023-03-03T06:10:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/w83l786ng.ko 2023-03-03T06:10:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-03-03T06:10:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-03-03T06:10:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/ 2023-03-03T06:10:08.396 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/algos/ 2023-03-03T06:10:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-03-03T06:10:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-03-03T06:10:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/ 2023-03-03T06:10:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-03-03T06:10:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-03-03T06:10:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-03-03T06:10:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-03-03T06:10:08.397 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-03-03T06:10:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-03-03T06:10:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-03-03T06:10:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-03-03T06:10:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-03-03T06:10:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-03-03T06:10:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-03-03T06:10:08.398 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-03-03T06:10:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-03-03T06:10:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-03-03T06:10:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-03-03T06:10:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-03-03T06:10:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-03-03T06:10:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-03-03T06:10:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-03-03T06:10:08.399 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-03-03T06:10:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-03-03T06:10:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-03-03T06:10:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-03-03T06:10:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-03-03T06:10:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-03-03T06:10:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-03-03T06:10:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-03-03T06:10:08.400 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-03-03T06:10:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-03-03T06:10:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-03-03T06:10:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-03-03T06:10:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-03-03T06:10:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-03-03T06:10:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-via.ko 2023-03-03T06:10:08.401 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-03-03T06:10:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-03-03T06:10:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-03-03T06:10:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/i2c-mux.ko 2023-03-03T06:10:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/i2c-smbus.ko 2023-03-03T06:10:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/i2c-stub.ko 2023-03-03T06:10:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/muxes/ 2023-03-03T06:10:08.402 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-03-03T06:10:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-03-03T06:10:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-03-03T06:10:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ 2023-03-03T06:10:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/ 2023-03-03T06:10:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 274945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/ib_cm.ko 2023-03-03T06:10:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 863201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/ib_core.ko 2023-03-03T06:10:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/ib_umad.ko 2023-03-03T06:10:08.403 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 321169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-03-03T06:10:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/iw_cm.ko 2023-03-03T06:10:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 276617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-03-03T06:10:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-03-03T06:10:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/ 2023-03-03T06:10:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/cxgb4/ 2023-03-03T06:10:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 499489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-03-03T06:10:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/mlx4/ 2023-03-03T06:10:08.404 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 425945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-03-03T06:10:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/mlx5/ 2023-03-03T06:10:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 734337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-03-03T06:10:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/mthca/ 2023-03-03T06:10:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 302809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-03-03T06:10:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/ocrdma/ 2023-03-03T06:10:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 157289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-03-03T06:10:08.405 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/usnic/ 2023-03-03T06:10:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 178337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-03-03T06:10:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/ 2023-03-03T06:10:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/ipoib/ 2023-03-03T06:10:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 272561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-03-03T06:10:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/iser/ 2023-03-03T06:10:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-03-03T06:10:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/isert/ 2023-03-03T06:10:08.406 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 119601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-03-03T06:10:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/srp/ 2023-03-03T06:10:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 134625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-03-03T06:10:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/srpt/ 2023-03-03T06:10:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 134249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-03-03T06:10:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/ 2023-03-03T06:10:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/evbug.ko 2023-03-03T06:10:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/ff-memless.ko 2023-03-03T06:10:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/gameport/ 2023-03-03T06:10:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-03-03T06:10:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/gameport/fm801-gp.ko 2023-03-03T06:10:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/gameport/gameport.ko 2023-03-03T06:10:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/gameport/lightning.ko 2023-03-03T06:10:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/gameport/ns558.ko 2023-03-03T06:10:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/joydev.ko 2023-03-03T06:10:08.408 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/ 2023-03-03T06:10:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-03-03T06:10:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-03-03T06:10:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-03-03T06:10:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-03-03T06:10:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-03-03T06:10:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-03-03T06:10:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/lkkbd.ko 2023-03-03T06:10:08.409 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/lm8323.ko 2023-03-03T06:10:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/lm8333.ko 2023-03-03T06:10:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-03-03T06:10:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-03-03T06:10:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-03-03T06:10:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-03-03T06:10:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/newtonkbd.ko 2023-03-03T06:10:08.410 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-03-03T06:10:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/qt1070.ko 2023-03-03T06:10:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/qt2160.ko 2023-03-03T06:10:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-03-03T06:10:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/stowaway.ko 2023-03-03T06:10:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/sunkbd.ko 2023-03-03T06:10:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-03-03T06:10:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-03-03T06:10:08.411 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-03-03T06:10:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/keyboard/xtkbd.ko 2023-03-03T06:10:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/matrix-keymap.ko 2023-03-03T06:10:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/ 2023-03-03T06:10:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/appletouch.ko 2023-03-03T06:10:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/bcm5974.ko 2023-03-03T06:10:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 126353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/cyapatp.ko 2023-03-03T06:10:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 95897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/elan_i2c.ko 2023-03-03T06:10:08.412 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/gpio_mouse.ko 2023-03-03T06:10:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 346145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/psmouse.ko 2023-03-03T06:10:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/sermouse.ko 2023-03-03T06:10:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-03-03T06:10:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-03-03T06:10:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-03-03T06:10:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/rmi4/ 2023-03-03T06:10:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 113481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/rmi4/rmi_core.ko 2023-03-03T06:10:08.413 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/ 2023-03-03T06:10:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/altera_ps2.ko 2023-03-03T06:10:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/arc_ps2.ko 2023-03-03T06:10:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/ct82c710.ko 2023-03-03T06:10:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-03-03T06:10:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/parkbd.ko 2023-03-03T06:10:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/pcips2.ko 2023-03-03T06:10:08.414 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/ps2mult.ko 2023-03-03T06:10:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/serio_raw.ko 2023-03-03T06:10:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/serio/serport.ko 2023-03-03T06:10:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/input/sparse-keymap.ko 2023-03-03T06:10:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/iommu/ 2023-03-03T06:10:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/iommu/amd/ 2023-03-03T06:10:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-03-03T06:10:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/ 2023-03-03T06:10:08.415 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-88pm860x.ko 2023-03-03T06:10:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-adp5520.ko 2023-03-03T06:10:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-bd2802.ko 2023-03-03T06:10:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-blinkm.ko 2023-03-03T06:10:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-da903x.ko 2023-03-03T06:10:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-da9052.ko 2023-03-03T06:10:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-gpio.ko 2023-03-03T06:10:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-lm3530.ko 2023-03-03T06:10:08.416 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-lm3533.ko 2023-03-03T06:10:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-lm355x.ko 2023-03-03T06:10:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-lm3642.ko 2023-03-03T06:10:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-lp3944.ko 2023-03-03T06:10:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-lp8788.ko 2023-03-03T06:10:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-max8997.ko 2023-03-03T06:10:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-mc13783.ko 2023-03-03T06:10:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-menf21bmc.ko 2023-03-03T06:10:08.417 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-pca9532.ko 2023-03-03T06:10:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-pca955x.ko 2023-03-03T06:10:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-pca963x.ko 2023-03-03T06:10:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-ss4200.ko 2023-03-03T06:10:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-tca6507.ko 2023-03-03T06:10:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-wm831x-status.ko 2023-03-03T06:10:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/leds-wm8350.ko 2023-03-03T06:10:08.418 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ 2023-03-03T06:10:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-03-03T06:10:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-03-03T06:10:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-03-03T06:10:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-03-03T06:10:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2023-03-03T06:10:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-03-03T06:10:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-03-03T06:10:08.419 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-03-03T06:10:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-03-03T06:10:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/ 2023-03-03T06:10:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/bcache/ 2023-03-03T06:10:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 650041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/bcache/bcache.ko 2023-03-03T06:10:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-bio-prison.ko 2023-03-03T06:10:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-bufio.ko 2023-03-03T06:10:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-cache-smq.ko 2023-03-03T06:10:08.420 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 156121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-cache.ko 2023-03-03T06:10:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 109385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-crypt.ko 2023-03-03T06:10:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-delay.ko 2023-03-03T06:10:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-era.ko 2023-03-03T06:10:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-flakey.ko 2023-03-03T06:10:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-log-userspace.ko 2023-03-03T06:10:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-log.ko 2023-03-03T06:10:08.421 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-mirror.ko 2023-03-03T06:10:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-multipath.ko 2023-03-03T06:10:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-queue-length.ko 2023-03-03T06:10:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-raid.ko 2023-03-03T06:10:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-region-hash.ko 2023-03-03T06:10:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-round-robin.ko 2023-03-03T06:10:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-service-time.ko 2023-03-03T06:10:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 111985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-snapshot.ko 2023-03-03T06:10:08.422 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-switch.ko 2023-03-03T06:10:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 186137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-thin-pool.ko 2023-03-03T06:10:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-verity.ko 2023-03-03T06:10:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/dm-zero.ko 2023-03-03T06:10:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/faulty.ko 2023-03-03T06:10:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/linear.ko 2023-03-03T06:10:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/multipath.ko 2023-03-03T06:10:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/persistent-data/ 2023-03-03T06:10:08.423 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 196865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-03-03T06:10:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/raid0.ko 2023-03-03T06:10:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 110553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/raid1.ko 2023-03-03T06:10:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 156729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/raid10.ko 2023-03-03T06:10:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 396433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/md/raid456.ko 2023-03-03T06:10:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/ 2023-03-03T06:10:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/ 2023-03-03T06:10:08.424 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 180969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptbase.ko 2023-03-03T06:10:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptctl.ko 2023-03-03T06:10:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptfc.ko 2023-03-03T06:10:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptlan.ko 2023-03-03T06:10:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 134233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptsas.ko 2023-03-03T06:10:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 79705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptscsih.ko 2023-03-03T06:10:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/message/fusion/mptspi.ko 2023-03-03T06:10:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/ 2023-03-03T06:10:08.425 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/88pm800.ko 2023-03-03T06:10:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/88pm805.ko 2023-03-03T06:10:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/88pm80x.ko 2023-03-03T06:10:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/arizona-i2c.ko 2023-03-03T06:10:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 127713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/arizona.ko 2023-03-03T06:10:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/bcm590xx.ko 2023-03-03T06:10:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/cros_ec_dev.ko 2023-03-03T06:10:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/dln2.ko 2023-03-03T06:10:08.426 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/htc-pasic3.ko 2023-03-03T06:10:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/janz-cmodio.ko 2023-03-03T06:10:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/kempld-core.ko 2023-03-03T06:10:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/lm3533-core.ko 2023-03-03T06:10:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-03-03T06:10:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/lp3943.ko 2023-03-03T06:10:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/lpc_ich.ko 2023-03-03T06:10:08.427 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/lpc_sch.ko 2023-03-03T06:10:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/max8907.ko 2023-03-03T06:10:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/mc13xxx-core.ko 2023-03-03T06:10:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-03-03T06:10:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/menf21bmc.ko 2023-03-03T06:10:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/pcf50633-adc.ko 2023-03-03T06:10:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/pcf50633-gpio.ko 2023-03-03T06:10:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/pcf50633.ko 2023-03-03T06:10:08.428 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-03-03T06:10:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/retu-mfd.ko 2023-03-03T06:10:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/si476x-core.ko 2023-03-03T06:10:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/sm501.ko 2023-03-03T06:10:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-03-03T06:10:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/tps65010.ko 2023-03-03T06:10:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/tps6507x.ko 2023-03-03T06:10:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/viperboard.ko 2023-03-03T06:10:08.429 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/vx855.ko 2023-03-03T06:10:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/wl1273-core.ko 2023-03-03T06:10:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/mfd/wm8994.ko 2023-03-03T06:10:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ 2023-03-03T06:10:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-03-03T06:10:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ad525x_dpot.ko 2023-03-03T06:10:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/altera-stapl/ 2023-03-03T06:10:08.430 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-03-03T06:10:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/apds9802als.ko 2023-03-03T06:10:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/apds990x.ko 2023-03-03T06:10:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/bh1770glc.ko 2023-03-03T06:10:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/c2port/ 2023-03-03T06:10:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-03-03T06:10:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/c2port/core.ko 2023-03-03T06:10:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/cb710/ 2023-03-03T06:10:08.431 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/cb710/cb710.ko 2023-03-03T06:10:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ds1682.ko 2023-03-03T06:10:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/dummy-irq.ko 2023-03-03T06:10:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/echo/ 2023-03-03T06:10:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/echo/echo.ko 2023-03-03T06:10:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/eeprom/ 2023-03-03T06:10:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/eeprom/at24.ko 2023-03-03T06:10:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/eeprom/eeprom.ko 2023-03-03T06:10:08.432 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-03-03T06:10:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/eeprom/max6875.ko 2023-03-03T06:10:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/enclosure.ko 2023-03-03T06:10:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/genwqe/ 2023-03-03T06:10:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 155889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-03-03T06:10:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/hmc6352.ko 2023-03-03T06:10:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/hpilo.ko 2023-03-03T06:10:08.433 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ibmasm/ 2023-03-03T06:10:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-03-03T06:10:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ics932s401.ko 2023-03-03T06:10:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/isl29003.ko 2023-03-03T06:10:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/isl29020.ko 2023-03-03T06:10:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/lis3lv02d/ 2023-03-03T06:10:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-03-03T06:10:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-03-03T06:10:08.434 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/mei/ 2023-03-03T06:10:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/mei/mei-me.ko 2023-03-03T06:10:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/mei/mei-txe.ko 2023-03-03T06:10:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 312217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/mei/mei.ko 2023-03-03T06:10:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/phantom.ko 2023-03-03T06:10:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ti-st/ 2023-03-03T06:10:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/ti-st/st_drv.ko 2023-03-03T06:10:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/tifm_7xx1.ko 2023-03-03T06:10:08.435 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/tifm_core.ko 2023-03-03T06:10:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/tsl2550.ko 2023-03-03T06:10:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/vmw_balloon.ko 2023-03-03T06:10:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/vmw_vmci/ 2023-03-03T06:10:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 182801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-03-03T06:10:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ 2023-03-03T06:10:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/bonding/ 2023-03-03T06:10:08.436 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 488953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/bonding/bonding.ko 2023-03-03T06:10:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/ 2023-03-03T06:10:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/b53/ 2023-03-03T06:10:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-03-03T06:10:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-03-03T06:10:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/mv88e6060.ko 2023-03-03T06:10:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/mv88e6xxx/ 2023-03-03T06:10:08.437 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 445361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-03-03T06:10:08.482 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/dummy.ko 2023-03-03T06:10:08.483 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/eql.ko 2023-03-03T06:10:08.483 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/ 2023-03-03T06:10:08.483 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/3com/ 2023-03-03T06:10:08.483 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-03-03T06:10:08.483 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-03-03T06:10:08.483 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 132169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-03-03T06:10:08.484 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-03-03T06:10:08.484 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/8390/ 2023-03-03T06:10:08.484 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/8390/8390.ko 2023-03-03T06:10:08.484 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-03-03T06:10:08.484 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-03-03T06:10:08.484 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-03-03T06:10:08.484 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/adaptec/ 2023-03-03T06:10:08.485 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-03-03T06:10:08.485 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/agere/ 2023-03-03T06:10:08.485 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-03-03T06:10:08.485 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/alteon/ 2023-03-03T06:10:08.485 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-03-03T06:10:08.485 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/altera/ 2023-03-03T06:10:08.485 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-03-03T06:10:08.485 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/amd/ 2023-03-03T06:10:08.486 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-03-03T06:10:08.486 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-03-03T06:10:08.486 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-03-03T06:10:08.486 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/ 2023-03-03T06:10:08.486 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/alx/ 2023-03-03T06:10:08.486 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-03-03T06:10:08.486 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-03-03T06:10:08.487 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-03-03T06:10:08.487 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-03-03T06:10:08.487 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-03-03T06:10:08.487 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atlx/ 2023-03-03T06:10:08.487 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-03-03T06:10:08.487 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-03-03T06:10:08.487 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/ 2023-03-03T06:10:08.487 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 79497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-03-03T06:10:08.488 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 176809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-03-03T06:10:08.488 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-03-03T06:10:08.488 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1535233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-03-03T06:10:08.488 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 130009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-03-03T06:10:08.488 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/genet/ 2023-03-03T06:10:08.488 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 133305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-03-03T06:10:08.488 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 316225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-03-03T06:10:08.489 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/brocade/ 2023-03-03T06:10:08.489 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/brocade/bna/ 2023-03-03T06:10:08.489 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 334513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-03-03T06:10:08.489 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/ 2023-03-03T06:10:08.489 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-03-03T06:10:08.489 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 194849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-03-03T06:10:08.489 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-03-03T06:10:08.489 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 353761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-03-03T06:10:08.490 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-03-03T06:10:08.490 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1061553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-03-03T06:10:08.490 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-03-03T06:10:08.490 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 169497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-03-03T06:10:08.490 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-03-03T06:10:08.490 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-03-03T06:10:08.490 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/cisco/ 2023-03-03T06:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/cisco/enic/ 2023-03-03T06:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 209577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-03-03T06:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/ 2023-03-03T06:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/ 2023-03-03T06:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-03-03T06:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-03-03T06:10:08.491 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 145593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-03-03T06:10:08.492 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-03-03T06:10:08.492 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-03-03T06:10:08.492 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-03-03T06:10:08.492 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dlink/ 2023-03-03T06:10:08.492 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-03-03T06:10:08.492 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-03-03T06:10:08.492 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/dnet.ko 2023-03-03T06:10:08.492 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/ec_bhf.ko 2023-03-03T06:10:08.493 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/emulex/ 2023-03-03T06:10:08.493 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/emulex/benet/ 2023-03-03T06:10:08.493 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 290761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-03-03T06:10:08.493 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/ethoc.ko 2023-03-03T06:10:08.493 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/fujitsu/ 2023-03-03T06:10:08.493 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-03-03T06:10:08.493 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ 2023-03-03T06:10:08.494 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 93185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/e100.ko 2023-03-03T06:10:08.494 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/e1000/ 2023-03-03T06:10:08.494 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 300377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-03-03T06:10:08.494 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/e1000e/ 2023-03-03T06:10:08.494 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 588441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-03-03T06:10:08.494 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/fm10k/ 2023-03-03T06:10:08.494 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 250753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-03-03T06:10:08.494 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/i40e/ 2023-03-03T06:10:08.495 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 979137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-03-03T06:10:08.495 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/iavf/ 2023-03-03T06:10:08.495 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 320785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-03-03T06:10:08.495 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/igb/ 2023-03-03T06:10:08.495 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 524993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-03-03T06:10:08.495 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/igbvf/ 2023-03-03T06:10:08.495 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 92905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-03-03T06:10:08.496 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ixgb/ 2023-03-03T06:10:08.496 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 126129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ixgb/ixgb.ko 2023-03-03T06:10:08.496 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-03-03T06:10:08.496 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 747761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-03-03T06:10:08.496 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-03-03T06:10:08.496 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 154305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-03-03T06:10:08.496 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/jme.ko 2023-03-03T06:10:08.496 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/marvell/ 2023-03-03T06:10:08.497 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-03-03T06:10:08.497 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-03-03T06:10:08.497 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-03-03T06:10:08.497 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/ 2023-03-03T06:10:08.497 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-03-03T06:10:08.497 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 745441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-03-03T06:10:08.497 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 277625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-03-03T06:10:08.497 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-03-03T06:10:08.498 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-03-03T06:10:08.498 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1048833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-03-03T06:10:08.498 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/micrel/ 2023-03-03T06:10:08.498 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-03-03T06:10:08.498 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-03-03T06:10:08.498 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-03-03T06:10:08.498 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 95801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-03-03T06:10:08.499 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/myricom/ 2023-03-03T06:10:08.499 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-03-03T06:10:08.499 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 115409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-03-03T06:10:08.499 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/natsemi/ 2023-03-03T06:10:08.499 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 76033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-03-03T06:10:08.499 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-03-03T06:10:08.499 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/neterion/ 2023-03-03T06:10:08.499 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 155121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-03-03T06:10:08.500 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/nvidia/ 2023-03-03T06:10:08.500 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 143961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-03-03T06:10:08.500 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/packetengines/ 2023-03-03T06:10:08.500 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-03-03T06:10:08.500 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-03-03T06:10:08.500 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/qlogic/ 2023-03-03T06:10:08.500 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-03-03T06:10:08.501 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 234649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-03-03T06:10:08.501 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-03-03T06:10:08.501 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-03-03T06:10:08.501 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 702361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-03-03T06:10:08.501 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/rdc/ 2023-03-03T06:10:08.501 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-03-03T06:10:08.501 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/realtek/ 2023-03-03T06:10:08.501 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-03-03T06:10:08.502 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-03-03T06:10:08.502 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-03-03T06:10:08.502 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 187265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-03-03T06:10:08.502 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/samsung/ 2023-03-03T06:10:08.502 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-03-03T06:10:08.502 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 104737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-03-03T06:10:08.502 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sfc/ 2023-03-03T06:10:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 848441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-03-03T06:10:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/silan/ 2023-03-03T06:10:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-03-03T06:10:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sis/ 2023-03-03T06:10:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-03-03T06:10:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-03-03T06:10:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/smsc/ 2023-03-03T06:10:08.503 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-03-03T06:10:08.504 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-03-03T06:10:08.504 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-03-03T06:10:08.504 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-03-03T06:10:08.504 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/stmicro/ 2023-03-03T06:10:08.504 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-03-03T06:10:08.504 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-03-03T06:10:08.504 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-03-03T06:10:08.504 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-03-03T06:10:08.505 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 476785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-03-03T06:10:08.505 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sun/ 2023-03-03T06:10:08.505 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-03-03T06:10:08.505 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 172713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sun/niu.ko 2023-03-03T06:10:08.505 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-03-03T06:10:08.505 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 86489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-03-03T06:10:08.505 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/tehuti/ 2023-03-03T06:10:08.506 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-03-03T06:10:08.506 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/ti/ 2023-03-03T06:10:08.506 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-03-03T06:10:08.506 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/via/ 2023-03-03T06:10:08.506 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-03-03T06:10:08.506 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-03-03T06:10:08.506 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/wiznet/ 2023-03-03T06:10:08.506 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-03-03T06:10:08.507 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-03-03T06:10:08.507 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/xircom/ 2023-03-03T06:10:08.507 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-03-03T06:10:08.507 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/fddi/ 2023-03-03T06:10:08.507 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/fddi/defxx.ko 2023-03-03T06:10:08.507 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/fddi/skfp/ 2023-03-03T06:10:08.507 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 223521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/fddi/skfp/skfp.ko 2023-03-03T06:10:08.507 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/geneve.ko 2023-03-03T06:10:08.508 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/hyperv/ 2023-03-03T06:10:08.508 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 238937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-03-03T06:10:08.508 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ieee802154/ 2023-03-03T06:10:08.508 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ieee802154/fakelb.ko 2023-03-03T06:10:08.508 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ifb.ko 2023-03-03T06:10:08.508 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ipvlan/ 2023-03-03T06:10:08.508 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 86849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-03-03T06:10:08.509 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/macvlan.ko 2023-03-03T06:10:08.509 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/macvtap.ko 2023-03-03T06:10:08.509 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/mdio/ 2023-03-03T06:10:08.509 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-03-03T06:10:08.509 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-03-03T06:10:08.509 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/mdio/mdio-gpio.ko 2023-03-03T06:10:08.509 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/mdio.ko 2023-03-03T06:10:08.509 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/mii.ko 2023-03-03T06:10:08.510 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/netconsole.ko 2023-03-03T06:10:08.510 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/nlmon.ko 2023-03-03T06:10:08.510 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/pcs/ 2023-03-03T06:10:08.510 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-03-03T06:10:08.510 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-03-03T06:10:08.510 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/ 2023-03-03T06:10:08.510 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/amd.ko 2023-03-03T06:10:08.511 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-03-03T06:10:08.511 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/bcm7xxx.ko 2023-03-03T06:10:08.511 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/bcm87xx.ko 2023-03-03T06:10:08.511 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/broadcom.ko 2023-03-03T06:10:08.511 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/cicada.ko 2023-03-03T06:10:08.511 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/davicom.ko 2023-03-03T06:10:08.511 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/et1011c.ko 2023-03-03T06:10:08.511 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/icplus.ko 2023-03-03T06:10:08.512 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/lxt.ko 2023-03-03T06:10:08.512 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/marvell.ko 2023-03-03T06:10:08.512 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/micrel.ko 2023-03-03T06:10:08.512 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/national.ko 2023-03-03T06:10:08.512 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 94009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/phylink.ko 2023-03-03T06:10:08.512 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/qsemi.ko 2023-03-03T06:10:08.512 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/realtek.ko 2023-03-03T06:10:08.512 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/smsc.ko 2023-03-03T06:10:08.513 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/ste10Xp.ko 2023-03-03T06:10:08.513 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/phy/vitesse.ko 2023-03-03T06:10:08.513 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/plip/ 2023-03-03T06:10:08.513 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/plip/plip.ko 2023-03-03T06:10:08.513 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/ 2023-03-03T06:10:08.513 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/bsd_comp.ko 2023-03-03T06:10:08.513 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/ppp_async.ko 2023-03-03T06:10:08.514 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-03-03T06:10:08.514 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-03-03T06:10:08.514 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-03-03T06:10:08.514 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/pppoe.ko 2023-03-03T06:10:08.514 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/pppox.ko 2023-03-03T06:10:08.514 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/ppp/pptp.ko 2023-03-03T06:10:08.514 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/sb1000.ko 2023-03-03T06:10:08.514 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/slip/ 2023-03-03T06:10:08.515 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/slip/slip.ko 2023-03-03T06:10:08.515 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/sungem_phy.ko 2023-03-03T06:10:08.515 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/tap.ko 2023-03-03T06:10:08.515 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/ 2023-03-03T06:10:08.515 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 105729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/team.ko 2023-03-03T06:10:08.515 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-03-03T06:10:08.515 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-03-03T06:10:08.515 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-03-03T06:10:08.516 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/team_mode_random.ko 2023-03-03T06:10:08.516 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-03-03T06:10:08.516 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/veth.ko 2023-03-03T06:10:08.516 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/vmxnet3/ 2023-03-03T06:10:08.516 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 121785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-03-03T06:10:08.516 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/vxlan/ 2023-03-03T06:10:08.516 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 251809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/vxlan/vxlan.ko 2023-03-03T06:10:08.517 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/xen-netback/ 2023-03-03T06:10:08.517 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 145569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-03-03T06:10:08.517 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ntb/ 2023-03-03T06:10:08.517 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ntb/ntb.ko 2023-03-03T06:10:08.517 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/nvme/ 2023-03-03T06:10:08.517 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/nvme/host/ 2023-03-03T06:10:08.517 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 260793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/nvme/host/nvme-core.ko 2023-03-03T06:10:08.517 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 105977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/nvme/host/nvme.ko 2023-03-03T06:10:08.518 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/parport/ 2023-03-03T06:10:08.518 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 130233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/parport/parport.ko 2023-03-03T06:10:08.518 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/parport/parport_ax88796.ko 2023-03-03T06:10:08.518 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/parport/parport_cs.ko 2023-03-03T06:10:08.518 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/parport/parport_pc.ko 2023-03-03T06:10:08.518 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/parport/parport_serial.ko 2023-03-03T06:10:08.518 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/ 2023-03-03T06:10:08.519 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/hotplug/ 2023-03-03T06:10:08.519 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-03-03T06:10:08.519 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-03-03T06:10:08.519 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-03-03T06:10:08.519 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/pci-stub.ko 2023-03-03T06:10:08.519 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pci/xen-pcifront.ko 2023-03-03T06:10:08.519 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/ 2023-03-03T06:10:08.519 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/i82092.ko 2023-03-03T06:10:08.520 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 146321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/pcmcia.ko 2023-03-03T06:10:08.520 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/pcmcia_core.ko 2023-03-03T06:10:08.520 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-03-03T06:10:08.520 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/pd6729.ko 2023-03-03T06:10:08.520 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 107353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pcmcia/yenta_socket.ko 2023-03-03T06:10:08.520 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/phy/ 2023-03-03T06:10:08.520 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/phy/broadcom/ 2023-03-03T06:10:08.520 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-03-03T06:10:08.521 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/phy/samsung/ 2023-03-03T06:10:08.521 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-03-03T06:10:08.521 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pinctrl/ 2023-03-03T06:10:08.521 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pinctrl/intel/ 2023-03-03T06:10:08.521 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-03-03T06:10:08.521 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/ 2023-03-03T06:10:08.521 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/ 2023-03-03T06:10:08.522 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-03-03T06:10:08.522 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros_ec.ko 2023-03-03T06:10:08.522 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-03-03T06:10:08.522 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-03-03T06:10:08.522 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-03-03T06:10:08.522 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-03-03T06:10:08.522 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-03-03T06:10:08.522 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/ 2023-03-03T06:10:08.523 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/acer-wmi.ko 2023-03-03T06:10:08.523 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/acerhdf.ko 2023-03-03T06:10:08.523 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/apple-gmux.ko 2023-03-03T06:10:08.523 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/asus-laptop.ko 2023-03-03T06:10:08.523 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-03-03T06:10:08.523 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 110601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/asus-wmi.ko 2023-03-03T06:10:08.523 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-03-03T06:10:08.523 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-03-03T06:10:08.524 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-03-03T06:10:08.524 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-03-03T06:10:08.524 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-03-03T06:10:08.524 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/hdaps.ko 2023-03-03T06:10:08.524 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/ibm_rtl.ko 2023-03-03T06:10:08.524 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/intel/ 2023-03-03T06:10:08.524 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-03-03T06:10:08.525 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-03-03T06:10:08.525 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/intel_ips.ko 2023-03-03T06:10:08.525 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/msi-wmi.ko 2023-03-03T06:10:08.525 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-03-03T06:10:08.525 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-03-03T06:10:08.525 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-03-03T06:10:08.525 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/samsung-q10.ko 2023-03-03T06:10:08.525 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-03-03T06:10:08.526 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-03-03T06:10:08.526 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-03-03T06:10:08.526 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-03-03T06:10:08.526 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/platform/x86/wmi.ko 2023-03-03T06:10:08.526 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/ 2023-03-03T06:10:08.526 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/ 2023-03-03T06:10:08.526 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30073 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/88pm860x_battery.ko 2023-03-03T06:10:08.527 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/88pm860x_charger.ko 2023-03-03T06:10:08.527 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/bq2415x_charger.ko 2023-03-03T06:10:08.527 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/bq24190_charger.ko 2023-03-03T06:10:08.527 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/bq24735-charger.ko 2023-03-03T06:10:08.527 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-03-03T06:10:08.527 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/da9030_battery.ko 2023-03-03T06:10:08.527 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/da9052-battery.ko 2023-03-03T06:10:08.527 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/ds2760_battery.ko 2023-03-03T06:10:08.528 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/ds2780_battery.ko 2023-03-03T06:10:08.528 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/ds2781_battery.ko 2023-03-03T06:10:08.528 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/ds2782_battery.ko 2023-03-03T06:10:08.528 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/gpio-charger.ko 2023-03-03T06:10:08.528 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/isp1704_charger.ko 2023-03-03T06:10:08.528 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/lp8727_charger.ko 2023-03-03T06:10:08.528 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/max14577_charger.ko 2023-03-03T06:10:08.528 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/max17040_battery.ko 2023-03-03T06:10:08.529 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/max17042_battery.ko 2023-03-03T06:10:08.529 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/max8903_charger.ko 2023-03-03T06:10:08.529 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/max8925_power.ko 2023-03-03T06:10:08.529 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/pcf50633-charger.ko 2023-03-03T06:10:08.529 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/pda_power.ko 2023-03-03T06:10:08.529 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/sbs-battery.ko 2023-03-03T06:10:08.529 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/test_power.ko 2023-03-03T06:10:08.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/tps65090-charger.ko 2023-03-03T06:10:08.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/wm831x_backup.ko 2023-03-03T06:10:08.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/wm831x_power.ko 2023-03-03T06:10:08.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/power/supply/wm8350_power.ko 2023-03-03T06:10:08.530 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/powercap/ 2023-03-03T06:10:08.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/powercap/intel_rapl_common.ko 2023-03-03T06:10:08.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-03-03T06:10:08.530 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pps/ 2023-03-03T06:10:08.531 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pps/clients/ 2023-03-03T06:10:08.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pps/clients/pps-gpio.ko 2023-03-03T06:10:08.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-03-03T06:10:08.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pps/clients/pps_parport.ko 2023-03-03T06:10:08.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/pps/pps_core.ko 2023-03-03T06:10:08.531 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ptp/ 2023-03-03T06:10:08.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ptp/ptp.ko 2023-03-03T06:10:08.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ptp/ptp_kvm.ko 2023-03-03T06:10:08.532 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/ 2023-03-03T06:10:08.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-88pm80x.ko 2023-03-03T06:10:08.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-88pm860x.ko 2023-03-03T06:10:08.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-bq32k.ko 2023-03-03T06:10:08.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-bq4802.ko 2023-03-03T06:10:08.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-da9052.ko 2023-03-03T06:10:08.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-da9055.ko 2023-03-03T06:10:08.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-da9063.ko 2023-03-03T06:10:08.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1286.ko 2023-03-03T06:10:08.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1307.ko 2023-03-03T06:10:08.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1374.ko 2023-03-03T06:10:08.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1511.ko 2023-03-03T06:10:08.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1553.ko 2023-03-03T06:10:08.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1672.ko 2023-03-03T06:10:08.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds1742.ko 2023-03-03T06:10:08.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds2404.ko 2023-03-03T06:10:08.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-ds3232.ko 2023-03-03T06:10:08.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-em3027.ko 2023-03-03T06:10:08.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-fm3130.ko 2023-03-03T06:10:08.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-isl12022.ko 2023-03-03T06:10:08.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-isl1208.ko 2023-03-03T06:10:08.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-lp8788.ko 2023-03-03T06:10:08.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-m41t80.ko 2023-03-03T06:10:08.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-m48t35.ko 2023-03-03T06:10:08.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-m48t59.ko 2023-03-03T06:10:08.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-m48t86.ko 2023-03-03T06:10:08.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-max6900.ko 2023-03-03T06:10:08.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-max8907.ko 2023-03-03T06:10:08.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-max8925.ko 2023-03-03T06:10:08.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-max8997.ko 2023-03-03T06:10:08.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-max8998.ko 2023-03-03T06:10:08.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-03-03T06:10:08.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-msm6242.ko 2023-03-03T06:10:08.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-palmas.ko 2023-03-03T06:10:08.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-03-03T06:10:08.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-pcf50633.ko 2023-03-03T06:10:08.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-03-03T06:10:08.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-03-03T06:10:08.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-03-03T06:10:08.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-03-03T06:10:08.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-rc5t583.ko 2023-03-03T06:10:08.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-03-03T06:10:08.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-03-03T06:10:08.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-03-03T06:10:08.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-rx8025.ko 2023-03-03T06:10:08.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-rx8581.ko 2023-03-03T06:10:08.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-s35390a.ko 2023-03-03T06:10:08.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-03-03T06:10:08.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-tps6586x.ko 2023-03-03T06:10:08.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-tps65910.ko 2023-03-03T06:10:08.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-v3020.ko 2023-03-03T06:10:08.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-wm831x.ko 2023-03-03T06:10:08.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-wm8350.ko 2023-03-03T06:10:08.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/rtc/rtc-x1205.ko 2023-03-03T06:10:08.539 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/ 2023-03-03T06:10:08.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/3w-9xxx.ko 2023-03-03T06:10:08.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/3w-sas.ko 2023-03-03T06:10:08.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/3w-xxxx.ko 2023-03-03T06:10:08.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 130513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/BusLogic.ko 2023-03-03T06:10:08.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/a100u2w.ko 2023-03-03T06:10:08.539 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aacraid/ 2023-03-03T06:10:08.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 278169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-03-03T06:10:08.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/advansys.ko 2023-03-03T06:10:08.540 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aic7xxx/ 2023-03-03T06:10:08.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 364593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-03-03T06:10:08.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 304337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-03-03T06:10:08.540 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aic94xx/ 2023-03-03T06:10:08.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 193489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-03-03T06:10:08.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/am53c974.ko 2023-03-03T06:10:08.541 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/arcmsr/ 2023-03-03T06:10:08.541 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 110193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-03-03T06:10:08.541 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/atp870u.ko 2023-03-03T06:10:08.541 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/be2iscsi/ 2023-03-03T06:10:08.541 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 263329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-03-03T06:10:08.541 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/bfa/ 2023-03-03T06:10:08.541 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1178769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/bfa/bfa.ko 2023-03-03T06:10:08.541 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/bnx2fc/ 2023-03-03T06:10:08.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 251577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-03-03T06:10:08.542 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/bnx2i/ 2023-03-03T06:10:08.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 121057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-03-03T06:10:08.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/ch.ko 2023-03-03T06:10:08.542 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/csiostor/ 2023-03-03T06:10:08.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 321433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-03-03T06:10:08.542 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/cxgbi/ 2023-03-03T06:10:08.542 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-03-03T06:10:08.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-03-03T06:10:08.543 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-03-03T06:10:08.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-03-03T06:10:08.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 153305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-03-03T06:10:08.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/dc395x.ko 2023-03-03T06:10:08.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/dmx3191d.ko 2023-03-03T06:10:08.543 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/esas2r/ 2023-03-03T06:10:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 212161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-03-03T06:10:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/esp_scsi.ko 2023-03-03T06:10:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/fcoe/ 2023-03-03T06:10:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-03-03T06:10:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 164201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-03-03T06:10:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/fdomain.ko 2023-03-03T06:10:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/fnic/ 2023-03-03T06:10:08.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 274297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/fnic/fnic.ko 2023-03-03T06:10:08.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 208345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/hpsa.ko 2023-03-03T06:10:08.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/hptiop.ko 2023-03-03T06:10:08.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/hv_storvsc.ko 2023-03-03T06:10:08.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/imm.ko 2023-03-03T06:10:08.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/initio.ko 2023-03-03T06:10:08.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 302617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/ipr.ko 2023-03-03T06:10:08.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/ips.ko 2023-03-03T06:10:08.545 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/isci/ 2023-03-03T06:10:08.546 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 328585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/isci/isci.ko 2023-03-03T06:10:08.546 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-03-03T06:10:08.546 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/iscsi_tcp.ko 2023-03-03T06:10:08.546 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/libfc/ 2023-03-03T06:10:08.546 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 325545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/libfc/libfc.ko 2023-03-03T06:10:08.546 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 147233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/libiscsi.ko 2023-03-03T06:10:08.546 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-03-03T06:10:08.547 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/libsas/ 2023-03-03T06:10:08.547 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 217481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/libsas/libsas.ko 2023-03-03T06:10:08.547 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/lpfc/ 2023-03-03T06:10:08.547 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2313865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-03-03T06:10:08.547 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/megaraid/ 2023-03-03T06:10:08.547 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-03-03T06:10:08.547 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-03-03T06:10:08.547 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 324825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-03-03T06:10:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 86001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/megaraid.ko 2023-03-03T06:10:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/mpt3sas/ 2023-03-03T06:10:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 699913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-03-03T06:10:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/mvsas/ 2023-03-03T06:10:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 121249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-03-03T06:10:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/mvumi.ko 2023-03-03T06:10:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pcmcia/ 2023-03-03T06:10:08.548 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-03-03T06:10:08.549 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-03-03T06:10:08.549 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-03-03T06:10:08.549 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-03-03T06:10:08.549 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pm8001/ 2023-03-03T06:10:08.549 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 405633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-03-03T06:10:08.549 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 131457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/pmcraid.ko 2023-03-03T06:10:08.549 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/ppa.ko 2023-03-03T06:10:08.550 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qla1280.ko 2023-03-03T06:10:08.550 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qla2xxx/ 2023-03-03T06:10:08.550 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1854193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-03-03T06:10:08.550 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-03-03T06:10:08.550 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qla4xxx/ 2023-03-03T06:10:08.550 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 596697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-03-03T06:10:08.550 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/qlogicfas408.ko 2023-03-03T06:10:08.550 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/raid_class.ko 2023-03-03T06:10:08.551 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 246129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/scsi_debug.ko 2023-03-03T06:10:08.551 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 173377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-03-03T06:10:08.551 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 276017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-03-03T06:10:08.551 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 94793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-03-03T06:10:08.551 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-03-03T06:10:08.551 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-03-03T06:10:08.551 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/ses.ko 2023-03-03T06:10:08.551 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 116161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/st.ko 2023-03-03T06:10:08.552 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/stex.ko 2023-03-03T06:10:08.552 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/sym53c8xx_2/ 2023-03-03T06:10:08.552 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 167465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-03-03T06:10:08.552 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/virtio_scsi.ko 2023-03-03T06:10:08.552 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-03-03T06:10:08.552 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/wd719x.ko 2023-03-03T06:10:08.552 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/scsi/xen-scsifront.ko 2023-03-03T06:10:08.553 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ssb/ 2023-03-03T06:10:08.553 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 130601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ssb/ssb.ko 2023-03-03T06:10:08.553 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/ 2023-03-03T06:10:08.553 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/iscsi/ 2023-03-03T06:10:08.553 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 761009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-03-03T06:10:08.553 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/loopback/ 2023-03-03T06:10:08.553 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/loopback/tcm_loop.ko 2023-03-03T06:10:08.553 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/target_core_file.ko 2023-03-03T06:10:08.554 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/target_core_iblock.ko 2023-03-03T06:10:08.554 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 937993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/target_core_mod.ko 2023-03-03T06:10:08.554 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/target_core_pscsi.ko 2023-03-03T06:10:08.554 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/target_core_user.ko 2023-03-03T06:10:08.554 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/tcm_fc/ 2023-03-03T06:10:08.554 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-03-03T06:10:08.554 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/ 2023-03-03T06:10:08.554 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/ 2023-03-03T06:10:08.555 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/ 2023-03-03T06:10:08.555 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-03-03T06:10:08.555 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-03-03T06:10:08.555 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-03-03T06:10:08.555 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-03-03T06:10:08.555 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-03-03T06:10:08.555 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-03-03T06:10:08.556 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-03-03T06:10:08.556 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-03-03T06:10:08.556 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-03-03T06:10:08.556 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-03-03T06:10:08.556 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-03-03T06:10:08.556 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-03-03T06:10:08.556 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/intel_menlow.ko 2023-03-03T06:10:08.556 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-03-03T06:10:08.557 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-03-03T06:10:08.557 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-03-03T06:10:08.557 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-03-03T06:10:08.557 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/ 2023-03-03T06:10:08.557 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/ipwireless/ 2023-03-03T06:10:08.557 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-03-03T06:10:08.557 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/moxa.ko 2023-03-03T06:10:08.558 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/mxser.ko 2023-03-03T06:10:08.558 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/n_gsm.ko 2023-03-03T06:10:08.558 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/n_hdlc.ko 2023-03-03T06:10:08.558 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/nozomi.ko 2023-03-03T06:10:08.558 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/ 2023-03-03T06:10:08.558 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/8250/ 2023-03-03T06:10:08.558 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-03-03T06:10:08.558 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-03-03T06:10:08.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-03-03T06:10:08.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/altera_uart.ko 2023-03-03T06:10:08.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/arc_uart.ko 2023-03-03T06:10:08.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-03-03T06:10:08.559 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/jsm/ 2023-03-03T06:10:08.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-03-03T06:10:08.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/rp2.ko 2023-03-03T06:10:08.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/serial/sc16is7xx.ko 2023-03-03T06:10:08.560 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/tty/synclink_gt.ko 2023-03-03T06:10:08.560 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ufs/ 2023-03-03T06:10:08.560 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ufs/core/ 2023-03-03T06:10:08.560 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 507913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-03-03T06:10:08.560 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ufs/host/ 2023-03-03T06:10:08.560 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-03-03T06:10:08.560 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-03-03T06:10:08.561 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/ 2023-03-03T06:10:08.561 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio.ko 2023-03-03T06:10:08.561 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_aec.ko 2023-03-03T06:10:08.561 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_cif.ko 2023-03-03T06:10:08.561 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_dmem_genirq.ko 2023-03-03T06:10:08.561 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_mf624.ko 2023-03-03T06:10:08.561 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_netx.ko 2023-03-03T06:10:08.561 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_pci_generic.ko 2023-03-03T06:10:08.562 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-03-03T06:10:08.562 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/uio/uio_sercos3.ko 2023-03-03T06:10:08.562 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/ 2023-03-03T06:10:08.562 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/c67x00/ 2023-03-03T06:10:08.562 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/c67x00/c67x00.ko 2023-03-03T06:10:08.562 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/chipidea/ 2023-03-03T06:10:08.562 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 148737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-03-03T06:10:08.563 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-03-03T06:10:08.563 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-03-03T06:10:08.563 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-03-03T06:10:08.563 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/class/ 2023-03-03T06:10:08.563 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 86289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/class/cdc-acm.ko 2023-03-03T06:10:08.563 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/class/cdc-wdm.ko 2023-03-03T06:10:08.563 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/class/usblp.ko 2023-03-03T06:10:08.563 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 93297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/class/usbtmc.ko 2023-03-03T06:10:08.564 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/common/ 2023-03-03T06:10:08.564 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/common/ulpi.ko 2023-03-03T06:10:08.564 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/dwc2/ 2023-03-03T06:10:08.564 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-03-03T06:10:08.564 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/dwc3/ 2023-03-03T06:10:08.564 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-03-03T06:10:08.564 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-03-03T06:10:08.564 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 308233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/dwc3/dwc3.ko 2023-03-03T06:10:08.565 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/ 2023-03-03T06:10:08.565 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/ 2023-03-03T06:10:08.565 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/u_ether.ko 2023-03-03T06:10:08.565 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/u_serial.ko 2023-03-03T06:10:08.565 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-03-03T06:10:08.565 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-03-03T06:10:08.565 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-03-03T06:10:08.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-03-03T06:10:08.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 92065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-03-03T06:10:08.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-03-03T06:10:08.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 121233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-03-03T06:10:08.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-03-03T06:10:08.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-03-03T06:10:08.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-03-03T06:10:08.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-03-03T06:10:08.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-03-03T06:10:08.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-03-03T06:10:08.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-03-03T06:10:08.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-03-03T06:10:08.567 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/ 2023-03-03T06:10:08.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-03-03T06:10:08.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-03-03T06:10:08.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-03-03T06:10:08.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21513 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-03-03T06:10:08.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-03-03T06:10:08.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-03-03T06:10:08.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-03-03T06:10:08.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-03-03T06:10:08.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-03-03T06:10:08.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-03-03T06:10:08.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-03-03T06:10:08.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-03-03T06:10:08.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-03-03T06:10:08.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-03-03T06:10:08.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 144601 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/libcomposite.ko 2023-03-03T06:10:08.569 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/ 2023-03-03T06:10:08.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-03-03T06:10:08.569 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/bdc/ 2023-03-03T06:10:08.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 158873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-03-03T06:10:08.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-03-03T06:10:08.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-03-03T06:10:08.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-03-03T06:10:08.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-03-03T06:10:08.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/net2272.ko 2023-03-03T06:10:08.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/net2280.ko 2023-03-03T06:10:08.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-03-03T06:10:08.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-03-03T06:10:08.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-03-03T06:10:08.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-03-03T06:10:08.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 149737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-03-03T06:10:08.571 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/ 2023-03-03T06:10:08.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/bcma-hcd.ko 2023-03-03T06:10:08.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/isp116x-hcd.ko 2023-03-03T06:10:08.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 88897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-03-03T06:10:08.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-03-03T06:10:08.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/sl811-hcd.ko 2023-03-03T06:10:08.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/sl811_cs.ko 2023-03-03T06:10:08.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/ssb-hcd.ko 2023-03-03T06:10:08.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 98497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/u132-hcd.ko 2023-03-03T06:10:08.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-03-03T06:10:08.572 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/image/ 2023-03-03T06:10:08.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/image/mdc800.ko 2023-03-03T06:10:08.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/image/microtek.ko 2023-03-03T06:10:08.573 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/ 2023-03-03T06:10:08.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/adutux.ko 2023-03-03T06:10:08.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/appledisplay.ko 2023-03-03T06:10:08.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-03-03T06:10:08.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/cytherm.ko 2023-03-03T06:10:08.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/ehset.ko 2023-03-03T06:10:08.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/emi26.ko 2023-03-03T06:10:08.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/emi62.ko 2023-03-03T06:10:08.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/ezusb.ko 2023-03-03T06:10:08.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/ftdi-elan.ko 2023-03-03T06:10:08.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/idmouse.ko 2023-03-03T06:10:08.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/iowarrior.ko 2023-03-03T06:10:08.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/isight_firmware.ko 2023-03-03T06:10:08.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/ldusb.ko 2023-03-03T06:10:08.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/legousbtower.ko 2023-03-03T06:10:08.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/lvstest.ko 2023-03-03T06:10:08.575 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/sisusbvga/ 2023-03-03T06:10:08.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-03-03T06:10:08.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/trancevibrator.ko 2023-03-03T06:10:08.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/usb3503.ko 2023-03-03T06:10:08.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/usblcd.ko 2023-03-03T06:10:08.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/usbsevseg.ko 2023-03-03T06:10:08.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/usbtest.ko 2023-03-03T06:10:08.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/uss720.ko 2023-03-03T06:10:08.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/misc/yurex.ko 2023-03-03T06:10:08.576 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/mon/ 2023-03-03T06:10:08.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/mon/usbmon.ko 2023-03-03T06:10:08.576 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/musb/ 2023-03-03T06:10:08.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 303201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/musb/musb_hdrc.ko 2023-03-03T06:10:08.577 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/phy/ 2023-03-03T06:10:08.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/phy/phy-generic.ko 2023-03-03T06:10:08.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-03-03T06:10:08.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/phy/phy-isp1301.ko 2023-03-03T06:10:08.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/phy/phy-tahvo.ko 2023-03-03T06:10:08.577 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ 2023-03-03T06:10:08.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/aircable.ko 2023-03-03T06:10:08.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ark3116.ko 2023-03-03T06:10:08.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/belkin_sa.ko 2023-03-03T06:10:08.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ch341.ko 2023-03-03T06:10:08.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/cp210x.ko 2023-03-03T06:10:08.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/cyberjack.ko 2023-03-03T06:10:08.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/cypress_m8.ko 2023-03-03T06:10:08.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-03-03T06:10:08.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/empeg.ko 2023-03-03T06:10:08.579 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/f81232.ko 2023-03-03T06:10:08.579 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 176201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-03-03T06:10:08.579 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/garmin_gps.ko 2023-03-03T06:10:08.579 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 138737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/io_edgeport.ko 2023-03-03T06:10:08.579 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 127865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/io_ti.ko 2023-03-03T06:10:08.579 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ipaq.ko 2023-03-03T06:10:08.579 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ipw.ko 2023-03-03T06:10:08.579 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ir-usb.ko 2023-03-03T06:10:08.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-03-03T06:10:08.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 109001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/keyspan.ko 2023-03-03T06:10:08.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-03-03T06:10:08.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-03-03T06:10:08.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/kobil_sct.ko 2023-03-03T06:10:08.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/mct_u232.ko 2023-03-03T06:10:08.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/metro-usb.ko 2023-03-03T06:10:08.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/mos7720.ko 2023-03-03T06:10:08.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/mos7840.ko 2023-03-03T06:10:08.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/mxuport.ko 2023-03-03T06:10:08.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/navman.ko 2023-03-03T06:10:08.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/omninet.ko 2023-03-03T06:10:08.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/opticon.ko 2023-03-03T06:10:08.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 186489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/option.ko 2023-03-03T06:10:08.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/oti6858.ko 2023-03-03T06:10:08.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/pl2303.ko 2023-03-03T06:10:08.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/qcaux.ko 2023-03-03T06:10:08.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/qcserial.ko 2023-03-03T06:10:08.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/quatech2.ko 2023-03-03T06:10:08.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/safe_serial.ko 2023-03-03T06:10:08.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/sierra.ko 2023-03-03T06:10:08.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/spcp8x5.ko 2023-03-03T06:10:08.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ssu100.ko 2023-03-03T06:10:08.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/symbolserial.ko 2023-03-03T06:10:08.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-03-03T06:10:08.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-03-03T06:10:08.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/usb_debug.ko 2023-03-03T06:10:08.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/usb_wwan.ko 2023-03-03T06:10:08.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 123553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/usbserial.ko 2023-03-03T06:10:08.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/visor.ko 2023-03-03T06:10:08.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/whiteheat.ko 2023-03-03T06:10:08.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/wishbone-serial.ko 2023-03-03T06:10:08.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/serial/xsens_mt.ko 2023-03-03T06:10:08.584 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ 2023-03-03T06:10:08.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/uas.ko 2023-03-03T06:10:08.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-alauda.ko 2023-03-03T06:10:08.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-cypress.ko 2023-03-03T06:10:08.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-datafab.ko 2023-03-03T06:10:08.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-03-03T06:10:08.585 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-freecom.ko 2023-03-03T06:10:08.585 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-isd200.ko 2023-03-03T06:10:08.585 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-03-03T06:10:08.585 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-karma.ko 2023-03-03T06:10:08.585 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-03-03T06:10:08.585 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-realtek.ko 2023-03-03T06:10:08.586 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-03-03T06:10:08.586 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-03-03T06:10:08.586 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/ums-usbat.ko 2023-03-03T06:10:08.586 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 177089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/storage/usb-storage.ko 2023-03-03T06:10:08.586 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/usbip/ 2023-03-03T06:10:08.586 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/usbip/usbip-core.ko 2023-03-03T06:10:08.586 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/usbip/usbip-host.ko 2023-03-03T06:10:08.587 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-03-03T06:10:08.587 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vfio/ 2023-03-03T06:10:08.587 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vfio/pci/ 2023-03-03T06:10:08.587 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 147089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-03-03T06:10:08.587 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-03-03T06:10:08.587 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 94817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vfio/vfio.ko 2023-03-03T06:10:08.587 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-03-03T06:10:08.587 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vhost/ 2023-03-03T06:10:08.588 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 98497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vhost/vhost.ko 2023-03-03T06:10:08.588 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vhost/vhost_iotlb.ko 2023-03-03T06:10:08.588 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vhost/vhost_net.ko 2023-03-03T06:10:08.588 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/vhost/vhost_scsi.ko 2023-03-03T06:10:08.588 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/ 2023-03-03T06:10:08.588 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/ 2023-03-03T06:10:08.588 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-03-03T06:10:08.589 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/aat2870_bl.ko 2023-03-03T06:10:08.589 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/adp5520_bl.ko 2023-03-03T06:10:08.589 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/adp8860_bl.ko 2023-03-03T06:10:08.589 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/adp8870_bl.ko 2023-03-03T06:10:08.589 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/apple_bl.ko 2023-03-03T06:10:08.589 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16721 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/as3711_bl.ko 2023-03-03T06:10:08.589 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/bd6107.ko 2023-03-03T06:10:08.589 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/da903x_bl.ko 2023-03-03T06:10:08.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/da9052_bl.ko 2023-03-03T06:10:08.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/gpio_backlight.ko 2023-03-03T06:10:08.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14337 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/kb3886_bl.ko 2023-03-03T06:10:08.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/lcd.ko 2023-03-03T06:10:08.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/lm3533_bl.ko 2023-03-03T06:10:08.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/lm3639_bl.ko 2023-03-03T06:10:08.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/lv5207lp.ko 2023-03-03T06:10:08.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/max8925_bl.ko 2023-03-03T06:10:08.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/pandora_bl.ko 2023-03-03T06:10:08.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-03-03T06:10:08.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/platform_lcd.ko 2023-03-03T06:10:08.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/video/backlight/wm831x_bl.ko 2023-03-03T06:10:08.591 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/ 2023-03-03T06:10:08.591 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/masters/ 2023-03-03T06:10:08.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/masters/ds1wm.ko 2023-03-03T06:10:08.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/masters/ds2482.ko 2023-03-03T06:10:08.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/masters/ds2490.ko 2023-03-03T06:10:08.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/masters/matrox_w1.ko 2023-03-03T06:10:08.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/masters/w1-gpio.ko 2023-03-03T06:10:08.592 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/ 2023-03-03T06:10:08.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-03-03T06:10:08.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-03-03T06:10:08.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-03-03T06:10:08.593 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-03-03T06:10:08.593 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-03-03T06:10:08.593 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-03-03T06:10:08.593 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-03-03T06:10:08.593 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-03-03T06:10:08.593 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-03-03T06:10:08.593 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_smem.ko 2023-03-03T06:10:08.593 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/slaves/w1_therm.ko 2023-03-03T06:10:08.594 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/w1/wire.ko 2023-03-03T06:10:08.594 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/ 2023-03-03T06:10:08.594 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/acquirewdt.ko 2023-03-03T06:10:08.594 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/advantechwdt.ko 2023-03-03T06:10:08.594 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-03-03T06:10:08.594 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-03-03T06:10:08.594 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/cpu5wdt.ko 2023-03-03T06:10:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/da9052_wdt.ko 2023-03-03T06:10:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/da9055_wdt.ko 2023-03-03T06:10:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/da9063_wdt.ko 2023-03-03T06:10:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/dw_wdt.ko 2023-03-03T06:10:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/eurotechwdt.ko 2023-03-03T06:10:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-03-03T06:10:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/hpwdt.ko 2023-03-03T06:10:08.595 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/i6300esb.ko 2023-03-03T06:10:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-03-03T06:10:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-03-03T06:10:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/ib700wdt.ko 2023-03-03T06:10:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/ibmasr.ko 2023-03-03T06:10:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-03-03T06:10:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-03-03T06:10:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/it87_wdt.ko 2023-03-03T06:10:08.596 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/kempld_wdt.ko 2023-03-03T06:10:08.597 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/machzwd.ko 2023-03-03T06:10:08.597 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/mena21_wdt.ko 2023-03-03T06:10:08.597 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-03-03T06:10:08.597 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/nv_tco.ko 2023-03-03T06:10:08.597 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-03-03T06:10:08.597 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/pc87413_wdt.ko 2023-03-03T06:10:08.597 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/pcwd_pci.ko 2023-03-03T06:10:08.597 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/pcwd_usb.ko 2023-03-03T06:10:08.598 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/retu_wdt.ko 2023-03-03T06:10:08.598 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-03-03T06:10:08.598 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-03-03T06:10:08.598 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-03-03T06:10:08.598 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/sc1200wdt.ko 2023-03-03T06:10:08.598 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-03-03T06:10:08.598 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-03-03T06:10:08.599 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/softdog.ko 2023-03-03T06:10:08.599 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/sp5100_tco.ko 2023-03-03T06:10:08.599 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/twl4030_wdt.ko 2023-03-03T06:10:08.599 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/via_wdt.ko 2023-03-03T06:10:08.599 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-03-03T06:10:08.599 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-03-03T06:10:08.599 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-03-03T06:10:08.599 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/wafer5823wdt.ko 2023-03-03T06:10:08.600 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/wdt_pci.ko 2023-03-03T06:10:08.600 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/wm831x_wdt.ko 2023-03-03T06:10:08.600 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/wm8350_wdt.ko 2023-03-03T06:10:08.600 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/watchdog/xen_wdt.ko 2023-03-03T06:10:08.600 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/ 2023-03-03T06:10:08.600 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23921 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-evtchn.ko 2023-03-03T06:10:08.600 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-gntalloc.ko 2023-03-03T06:10:08.600 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-gntdev.ko 2023-03-03T06:10:08.601 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-pciback/ 2023-03-03T06:10:08.601 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 179209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-03-03T06:10:08.601 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-privcmd.ko 2023-03-03T06:10:08.601 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xen-scsiback.ko 2023-03-03T06:10:08.601 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xenfs/ 2023-03-03T06:10:08.601 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/drivers/xen/xenfs/xenfs.ko 2023-03-03T06:10:08.601 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ 2023-03-03T06:10:08.601 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/9p/ 2023-03-03T06:10:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 170937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/9p/9p.ko 2023-03-03T06:10:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/adfs/ 2023-03-03T06:10:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/adfs/adfs.ko 2023-03-03T06:10:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/affs/ 2023-03-03T06:10:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 162665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/affs/affs.ko 2023-03-03T06:10:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/afs/ 2023-03-03T06:10:08.602 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 886529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/afs/kafs.ko 2023-03-03T06:10:08.603 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/autofs/ 2023-03-03T06:10:08.603 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/autofs/autofs4.ko 2023-03-03T06:10:08.603 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/befs/ 2023-03-03T06:10:08.603 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64073 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/befs/befs.ko 2023-03-03T06:10:08.603 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/bfs/ 2023-03-03T06:10:08.603 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/bfs/bfs.ko 2023-03-03T06:10:08.603 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/binfmt_misc.ko 2023-03-03T06:10:08.603 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/btrfs/ 2023-03-03T06:10:08.604 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 3478321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/btrfs/btrfs.ko 2023-03-03T06:10:08.604 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/cachefiles/ 2023-03-03T06:10:08.604 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 293745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/cachefiles/cachefiles.ko 2023-03-03T06:10:08.604 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ceph/ 2023-03-03T06:10:08.604 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1240393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ceph/ceph.ko 2023-03-03T06:10:08.604 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/cifs/ 2023-03-03T06:10:08.604 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2600641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/cifs/cifs.ko 2023-03-03T06:10:08.604 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/coda/ 2023-03-03T06:10:08.605 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/coda/coda.ko 2023-03-03T06:10:08.605 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/configfs/ 2023-03-03T06:10:08.605 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 105777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/configfs/configfs.ko 2023-03-03T06:10:08.605 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/cramfs/ 2023-03-03T06:10:08.605 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/cramfs/cramfs.ko 2023-03-03T06:10:08.605 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/dlm/ 2023-03-03T06:10:08.605 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 637665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/dlm/dlm.ko 2023-03-03T06:10:08.606 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/efs/ 2023-03-03T06:10:08.606 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/efs/efs.ko 2023-03-03T06:10:08.606 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/f2fs/ 2023-03-03T06:10:08.606 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1485049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/f2fs/f2fs.ko 2023-03-03T06:10:08.633 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/fat/ 2023-03-03T06:10:08.633 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/fat/msdos.ko 2023-03-03T06:10:08.634 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/freevxfs/ 2023-03-03T06:10:08.634 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/freevxfs/freevxfs.ko 2023-03-03T06:10:08.634 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/fscache/ 2023-03-03T06:10:08.634 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 204017 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/fscache/fscache.ko 2023-03-03T06:10:08.634 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/fuse/ 2023-03-03T06:10:08.634 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/fuse/cuse.ko 2023-03-03T06:10:08.634 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/gfs2/ 2023-03-03T06:10:08.635 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1234073 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/gfs2/gfs2.ko 2023-03-03T06:10:08.635 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/hfs/ 2023-03-03T06:10:08.635 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 144753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/hfs/hfs.ko 2023-03-03T06:10:08.635 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/hfsplus/ 2023-03-03T06:10:08.635 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 229193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/hfsplus/hfsplus.ko 2023-03-03T06:10:08.635 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/hpfs/ 2023-03-03T06:10:08.635 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 200097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/hpfs/hpfs.ko 2023-03-03T06:10:08.636 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/isofs/ 2023-03-03T06:10:08.636 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/isofs/isofs.ko 2023-03-03T06:10:08.636 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/jfs/ 2023-03-03T06:10:08.636 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 424905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/jfs/jfs.ko 2023-03-03T06:10:08.636 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/lockd/ 2023-03-03T06:10:08.636 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 279945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/lockd/lockd.ko 2023-03-03T06:10:08.636 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/minix/ 2023-03-03T06:10:08.636 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 88993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/minix/minix.ko 2023-03-03T06:10:08.637 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/netfs/ 2023-03-03T06:10:08.637 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 113025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/netfs/netfs.ko 2023-03-03T06:10:08.637 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/ 2023-03-03T06:10:08.637 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/blocklayout/ 2023-03-03T06:10:08.637 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-03-03T06:10:08.637 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/filelayout/ 2023-03-03T06:10:08.637 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-03-03T06:10:08.638 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/flexfilelayout/ 2023-03-03T06:10:08.638 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-03-03T06:10:08.638 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1072689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/nfs.ko 2023-03-03T06:10:08.638 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/nfsv2.ko 2023-03-03T06:10:08.638 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/nfsv3.ko 2023-03-03T06:10:08.638 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2057697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs/nfsv4.ko 2023-03-03T06:10:08.638 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs_common/ 2023-03-03T06:10:08.638 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs_common/grace.ko 2023-03-03T06:10:08.639 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfs_common/nfs_acl.ko 2023-03-03T06:10:08.639 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfsd/ 2023-03-03T06:10:08.639 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1585681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nfsd/nfsd.ko 2023-03-03T06:10:08.639 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nilfs2/ 2023-03-03T06:10:08.639 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 586265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nilfs2/nilfs2.ko 2023-03-03T06:10:08.639 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/ 2023-03-03T06:10:08.639 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-celtic.ko 2023-03-03T06:10:08.640 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-centeuro.ko 2023-03-03T06:10:08.640 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-croatian.ko 2023-03-03T06:10:08.640 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-cyrillic.ko 2023-03-03T06:10:08.640 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-gaelic.ko 2023-03-03T06:10:08.640 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-greek.ko 2023-03-03T06:10:08.640 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-iceland.ko 2023-03-03T06:10:08.640 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-inuit.ko 2023-03-03T06:10:08.640 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-roman.ko 2023-03-03T06:10:08.641 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-romanian.ko 2023-03-03T06:10:08.641 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/mac-turkish.ko 2023-03-03T06:10:08.641 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_ascii.ko 2023-03-03T06:10:08.641 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp1250.ko 2023-03-03T06:10:08.641 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp1251.ko 2023-03-03T06:10:08.641 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp1255.ko 2023-03-03T06:10:08.641 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp737.ko 2023-03-03T06:10:08.641 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp775.ko 2023-03-03T06:10:08.642 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp850.ko 2023-03-03T06:10:08.642 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp852.ko 2023-03-03T06:10:08.642 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp855.ko 2023-03-03T06:10:08.642 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp857.ko 2023-03-03T06:10:08.642 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp860.ko 2023-03-03T06:10:08.642 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp861.ko 2023-03-03T06:10:08.642 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp862.ko 2023-03-03T06:10:08.643 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp863.ko 2023-03-03T06:10:08.643 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp864.ko 2023-03-03T06:10:08.643 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp865.ko 2023-03-03T06:10:08.643 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp866.ko 2023-03-03T06:10:08.643 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp869.ko 2023-03-03T06:10:08.643 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp874.ko 2023-03-03T06:10:08.643 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 93649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp932.ko 2023-03-03T06:10:08.643 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 142185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp936.ko 2023-03-03T06:10:08.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 165321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp949.ko 2023-03-03T06:10:08.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 117625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_cp950.ko 2023-03-03T06:10:08.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_euc-jp.ko 2023-03-03T06:10:08.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-1.ko 2023-03-03T06:10:08.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-13.ko 2023-03-03T06:10:08.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-14.ko 2023-03-03T06:10:08.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-15.ko 2023-03-03T06:10:08.644 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-2.ko 2023-03-03T06:10:08.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-3.ko 2023-03-03T06:10:08.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-4.ko 2023-03-03T06:10:08.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-5.ko 2023-03-03T06:10:08.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-6.ko 2023-03-03T06:10:08.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-7.ko 2023-03-03T06:10:08.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_iso8859-9.ko 2023-03-03T06:10:08.645 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_koi8-r.ko 2023-03-03T06:10:08.646 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_koi8-ru.ko 2023-03-03T06:10:08.646 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_koi8-u.ko 2023-03-03T06:10:08.646 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/nls/nls_utf8.ko 2023-03-03T06:10:08.646 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ntfs/ 2023-03-03T06:10:08.646 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 219449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ntfs/ntfs.ko 2023-03-03T06:10:08.646 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/ 2023-03-03T06:10:08.646 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/cluster/ 2023-03-03T06:10:08.646 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 242329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-03-03T06:10:08.647 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/dlm/ 2023-03-03T06:10:08.647 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 439761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-03-03T06:10:08.647 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/dlmfs/ 2023-03-03T06:10:08.647 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-03-03T06:10:08.647 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2625897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/ocfs2.ko 2023-03-03T06:10:08.647 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-03-03T06:10:08.647 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-03-03T06:10:08.647 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-03-03T06:10:08.648 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/omfs/ 2023-03-03T06:10:08.648 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/omfs/omfs.ko 2023-03-03T06:10:08.648 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/overlayfs/ 2023-03-03T06:10:08.648 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 334953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/overlayfs/overlay.ko 2023-03-03T06:10:08.648 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/pstore/ 2023-03-03T06:10:08.648 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/pstore/ramoops.ko 2023-03-03T06:10:08.648 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/qnx4/ 2023-03-03T06:10:08.649 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/qnx4/qnx4.ko 2023-03-03T06:10:08.649 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/qnx6/ 2023-03-03T06:10:08.649 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/qnx6/qnx6.ko 2023-03-03T06:10:08.649 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/quota/ 2023-03-03T06:10:08.649 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/quota/quota_tree.ko 2023-03-03T06:10:08.649 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/quota/quota_v1.ko 2023-03-03T06:10:08.649 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21521 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/quota/quota_v2.ko 2023-03-03T06:10:08.649 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/reiserfs/ 2023-03-03T06:10:08.650 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 530121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/reiserfs/reiserfs.ko 2023-03-03T06:10:08.650 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/romfs/ 2023-03-03T06:10:08.650 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/romfs/romfs.ko 2023-03-03T06:10:08.650 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/smbfs_common/ 2023-03-03T06:10:08.650 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/smbfs_common/cifs_arc4.ko 2023-03-03T06:10:08.650 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/smbfs_common/cifs_md4.ko 2023-03-03T06:10:08.650 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/squashfs/ 2023-03-03T06:10:08.650 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 134969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/squashfs/squashfs.ko 2023-03-03T06:10:08.651 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/sysv/ 2023-03-03T06:10:08.651 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 92681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/sysv/sysv.ko 2023-03-03T06:10:08.651 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/udf/ 2023-03-03T06:10:08.651 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 253265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/udf/udf.ko 2023-03-03T06:10:08.651 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ufs/ 2023-03-03T06:10:08.651 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 162185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/ufs/ufs.ko 2023-03-03T06:10:08.651 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/xfs/ 2023-03-03T06:10:08.651 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 3960561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/fs/xfs/xfs.ko 2023-03-03T06:10:08.652 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/kernel/ 2023-03-03T06:10:08.652 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/kernel/locking/ 2023-03-03T06:10:08.652 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/kernel/locking/locktorture.ko 2023-03-03T06:10:08.652 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/kernel/time/ 2023-03-03T06:10:08.652 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/kernel/time/test_udelay.ko 2023-03-03T06:10:08.652 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/kernel/torture.ko 2023-03-03T06:10:08.652 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/ 2023-03-03T06:10:08.653 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/crc-itu-t.ko 2023-03-03T06:10:08.653 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 5729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/crc7.ko 2023-03-03T06:10:08.653 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/crc8.ko 2023-03-03T06:10:08.653 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/crypto/ 2023-03-03T06:10:08.653 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/crypto/libarc4.ko 2023-03-03T06:10:08.653 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/crypto/libdes.ko 2023-03-03T06:10:08.653 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/interval_tree_test.ko 2023-03-03T06:10:08.653 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/libcrc32c.ko 2023-03-03T06:10:08.654 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/lru_cache.ko 2023-03-03T06:10:08.654 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/lz4/ 2023-03-03T06:10:08.654 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/lz4/lz4_compress.ko 2023-03-03T06:10:08.654 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/lz4/lz4hc_compress.ko 2023-03-03T06:10:08.654 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/math/ 2023-03-03T06:10:08.654 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 5897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/math/cordic.ko 2023-03-03T06:10:08.654 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/memory-notifier-error-inject.ko 2023-03-03T06:10:08.655 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/notifier-error-inject.ko 2023-03-03T06:10:08.655 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/percpu_test.ko 2023-03-03T06:10:08.655 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/pm-notifier-error-inject.ko 2023-03-03T06:10:08.655 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/raid6/ 2023-03-03T06:10:08.655 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 141425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/raid6/raid6_pq.ko 2023-03-03T06:10:08.655 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/rbtree_test.ko 2023-03-03T06:10:08.655 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/reed_solomon/ 2023-03-03T06:10:08.655 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/reed_solomon/reed_solomon.ko 2023-03-03T06:10:08.656 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/test-kstrtox.ko 2023-03-03T06:10:08.656 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/test-string_helpers.ko 2023-03-03T06:10:08.656 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 4759153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/test_bpf.ko 2023-03-03T06:10:08.656 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49489 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/test_firmware.ko 2023-03-03T06:10:08.656 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 5873 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/test_module.ko 2023-03-03T06:10:08.656 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/test_user_copy.ko 2023-03-03T06:10:08.656 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/ts_bm.ko 2023-03-03T06:10:08.656 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/ts_fsm.ko 2023-03-03T06:10:08.657 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/ts_kmp.ko 2023-03-03T06:10:08.657 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/xz/ 2023-03-03T06:10:08.657 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/lib/xz/xz_dec_test.ko 2023-03-03T06:10:08.657 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/mm/ 2023-03-03T06:10:08.657 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/mm/hwpoison-inject.ko 2023-03-03T06:10:08.657 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ 2023-03-03T06:10:08.657 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/ 2023-03-03T06:10:08.658 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/6lowpan.ko 2023-03-03T06:10:08.658 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_dest.ko 2023-03-03T06:10:08.658 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_fragment.ko 2023-03-03T06:10:08.658 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_hop.ko 2023-03-03T06:10:08.658 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_ipv6.ko 2023-03-03T06:10:08.658 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_mobility.ko 2023-03-03T06:10:08.658 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_routing.ko 2023-03-03T06:10:08.658 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/6lowpan/nhc_udp.ko 2023-03-03T06:10:08.659 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/802/ 2023-03-03T06:10:08.659 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/802/garp.ko 2023-03-03T06:10:08.659 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/802/mrp.ko 2023-03-03T06:10:08.659 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/802/p8022.ko 2023-03-03T06:10:08.659 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/802/psnap.ko 2023-03-03T06:10:08.659 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/802/stp.ko 2023-03-03T06:10:08.659 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/8021q/ 2023-03-03T06:10:08.659 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 86249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/8021q/8021q.ko 2023-03-03T06:10:08.660 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/9p/ 2023-03-03T06:10:08.660 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 178153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/9p/9pnet.ko 2023-03-03T06:10:08.660 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/9p/9pnet_fd.ko 2023-03-03T06:10:08.660 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/9p/9pnet_rdma.ko 2023-03-03T06:10:08.660 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/9p/9pnet_virtio.ko 2023-03-03T06:10:08.660 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/batman-adv/ 2023-03-03T06:10:08.660 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 651577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/batman-adv/batman-adv.ko 2023-03-03T06:10:08.660 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/ 2023-03-03T06:10:08.661 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/br_netfilter.ko 2023-03-03T06:10:08.661 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 755113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/bridge.ko 2023-03-03T06:10:08.661 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ 2023-03-03T06:10:08.661 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-03-03T06:10:08.661 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_among.ko 2023-03-03T06:10:08.661 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-03-03T06:10:08.661 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-03-03T06:10:08.662 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-03-03T06:10:08.662 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8249 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-03-03T06:10:08.662 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-03-03T06:10:08.662 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-03-03T06:10:08.662 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_log.ko 2023-03-03T06:10:08.662 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-03-03T06:10:08.662 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-03-03T06:10:08.662 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-03-03T06:10:08.663 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-03-03T06:10:08.663 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-03-03T06:10:08.663 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-03-03T06:10:08.663 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-03-03T06:10:08.663 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-03-03T06:10:08.663 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-03-03T06:10:08.663 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-03-03T06:10:08.663 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-03-03T06:10:08.664 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 79953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/ebtables.ko 2023-03-03T06:10:08.664 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-03-03T06:10:08.664 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-03-03T06:10:08.664 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ceph/ 2023-03-03T06:10:08.664 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1047633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ceph/libceph.ko 2023-03-03T06:10:08.664 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/core/ 2023-03-03T06:10:08.664 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 129729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/core/pktgen.ko 2023-03-03T06:10:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dccp/ 2023-03-03T06:10:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 192145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dccp/dccp.ko 2023-03-03T06:10:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dccp/dccp_diag.ko 2023-03-03T06:10:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dccp/dccp_ipv4.ko 2023-03-03T06:10:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dccp/dccp_ipv6.ko 2023-03-03T06:10:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dsa/ 2023-03-03T06:10:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 276553 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dsa/dsa_core.ko 2023-03-03T06:10:08.665 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dsa/tag_brcm.ko 2023-03-03T06:10:08.666 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dsa/tag_dsa.ko 2023-03-03T06:10:08.666 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dsa/tag_none.ko 2023-03-03T06:10:08.666 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/dsa/tag_trailer.ko 2023-03-03T06:10:08.666 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/hsr/ 2023-03-03T06:10:08.666 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99145 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/hsr/hsr.ko 2023-03-03T06:10:08.666 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ieee802154/ 2023-03-03T06:10:08.666 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ieee802154/6lowpan/ 2023-03-03T06:10:08.666 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-03-03T06:10:08.667 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 214049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ieee802154/ieee802154.ko 2023-03-03T06:10:08.667 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ieee802154/ieee802154_socket.ko 2023-03-03T06:10:08.667 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ 2023-03-03T06:10:08.667 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ah4.ko 2023-03-03T06:10:08.667 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/esp4.ko 2023-03-03T06:10:08.667 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/fou.ko 2023-03-03T06:10:08.667 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/gre.ko 2023-03-03T06:10:08.668 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/inet_diag.ko 2023-03-03T06:10:08.668 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ip_gre.ko 2023-03-03T06:10:08.668 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ip_tunnel.ko 2023-03-03T06:10:08.668 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ip_vti.ko 2023-03-03T06:10:08.668 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ipcomp.ko 2023-03-03T06:10:08.668 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/ipip.ko 2023-03-03T06:10:08.668 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ 2023-03-03T06:10:08.668 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-03-03T06:10:08.669 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8385 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-03-03T06:10:08.669 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-03-03T06:10:08.669 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54849 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ip_tables.ko 2023-03-03T06:10:08.669 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ipt_CLUSTERIP.ko 2023-03-03T06:10:08.669 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-03-03T06:10:08.669 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-03-03T06:10:08.669 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-03-03T06:10:08.669 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-03-03T06:10:08.670 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-03-03T06:10:08.670 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-03-03T06:10:08.670 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-03-03T06:10:08.670 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-03-03T06:10:08.670 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-03-03T06:10:08.670 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-03-03T06:10:08.670 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-03-03T06:10:08.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-03-03T06:10:08.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-03-03T06:10:08.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-03-03T06:10:08.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-03-03T06:10:08.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-03-03T06:10:08.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-03-03T06:10:08.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-03-03T06:10:08.671 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-03-03T06:10:08.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-03-03T06:10:08.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-03-03T06:10:08.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_bic.ko 2023-03-03T06:10:08.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_dctcp.ko 2023-03-03T06:10:08.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_diag.ko 2023-03-03T06:10:08.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_highspeed.ko 2023-03-03T06:10:08.672 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_htcp.ko 2023-03-03T06:10:08.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_hybla.ko 2023-03-03T06:10:08.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_illinois.ko 2023-03-03T06:10:08.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_lp.ko 2023-03-03T06:10:08.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7841 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_scalable.ko 2023-03-03T06:10:08.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_vegas.ko 2023-03-03T06:10:08.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_veno.ko 2023-03-03T06:10:08.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10465 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_westwood.ko 2023-03-03T06:10:08.673 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9689 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tcp_yeah.ko 2023-03-03T06:10:08.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/tunnel4.ko 2023-03-03T06:10:08.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15697 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/udp_diag.ko 2023-03-03T06:10:08.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40105 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/udp_tunnel.ko 2023-03-03T06:10:08.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-03-03T06:10:08.674 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ 2023-03-03T06:10:08.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26353 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ah6.ko 2023-03-03T06:10:08.674 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/esp6.ko 2023-03-03T06:10:08.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/fou6.ko 2023-03-03T06:10:08.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 96425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ip6_gre.ko 2023-03-03T06:10:08.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ip6_tunnel.ko 2023-03-03T06:10:08.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-03-03T06:10:08.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ip6_vti.ko 2023-03-03T06:10:08.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/ipcomp6.ko 2023-03-03T06:10:08.675 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/mip6.ko 2023-03-03T06:10:08.676 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ 2023-03-03T06:10:08.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55449 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-03-03T06:10:08.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-03-03T06:10:08.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-03-03T06:10:08.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15657 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-03-03T06:10:08.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-03-03T06:10:08.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-03-03T06:10:08.676 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-03-03T06:10:08.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-03-03T06:10:08.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-03-03T06:10:08.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-03-03T06:10:08.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-03-03T06:10:08.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-03-03T06:10:08.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-03-03T06:10:08.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-03-03T06:10:08.677 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-03-03T06:10:08.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11097 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-03-03T06:10:08.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-03-03T06:10:08.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-03-03T06:10:08.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-03-03T06:10:08.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-03-03T06:10:08.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-03-03T06:10:08.678 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-03-03T06:10:08.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-03-03T06:10:08.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-03-03T06:10:08.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-03-03T06:10:08.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/sit.ko 2023-03-03T06:10:08.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/tunnel6.ko 2023-03-03T06:10:08.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-03-03T06:10:08.679 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/key/ 2023-03-03T06:10:08.679 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 92481 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/key/af_key.ko 2023-03-03T06:10:08.680 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/ 2023-03-03T06:10:08.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 133505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_core.ko 2023-03-03T06:10:08.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_debugfs.ko 2023-03-03T06:10:08.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_eth.ko 2023-03-03T06:10:08.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_ip.ko 2023-03-03T06:10:08.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36281 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_ip6.ko 2023-03-03T06:10:08.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_netlink.ko 2023-03-03T06:10:08.680 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/l2tp/l2tp_ppp.ko 2023-03-03T06:10:08.681 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/lapb/ 2023-03-03T06:10:08.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46593 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/lapb/lapb.ko 2023-03-03T06:10:08.681 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/llc/ 2023-03-03T06:10:08.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/llc/llc.ko 2023-03-03T06:10:08.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 266329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/llc/llc2.ko 2023-03-03T06:10:08.681 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/mac802154/ 2023-03-03T06:10:08.681 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 238257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/mac802154/mac802154.ko 2023-03-03T06:10:08.682 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/mpls/ 2023-03-03T06:10:08.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/mpls/mpls_gso.ko 2023-03-03T06:10:08.682 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ 2023-03-03T06:10:08.682 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ 2023-03-03T06:10:08.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 135785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set.ko 2023-03-03T06:10:08.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-03-03T06:10:08.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-03-03T06:10:08.682 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-03-03T06:10:08.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-03-03T06:10:08.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-03-03T06:10:08.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 104065 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-03-03T06:10:08.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 104529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-03-03T06:10:08.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 112081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-03-03T06:10:08.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60705 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-03-03T06:10:08.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-03-03T06:10:08.683 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 112361 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-03-03T06:10:08.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 111865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-03-03T06:10:08.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 110785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-03-03T06:10:08.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 113737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-03-03T06:10:08.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-03-03T06:10:08.684 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ 2023-03-03T06:10:08.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 407209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-03-03T06:10:08.684 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-03-03T06:10:08.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-03-03T06:10:08.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17225 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-03-03T06:10:08.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-03-03T06:10:08.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-03-03T06:10:08.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-03-03T06:10:08.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-03-03T06:10:08.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-03-03T06:10:08.685 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-03-03T06:10:08.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-03-03T06:10:08.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-03-03T06:10:08.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-03-03T06:10:08.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-03-03T06:10:08.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conncount.ko 2023-03-03T06:10:08.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 429161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack.ko 2023-03-03T06:10:08.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-03-03T06:10:08.686 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9345 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-03-03T06:10:08.687 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-03-03T06:10:08.687 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 116537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-03-03T06:10:08.687 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-03-03T06:10:08.687 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-03-03T06:10:08.687 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 105961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-03-03T06:10:08.687 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32673 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-03-03T06:10:08.687 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-03-03T06:10:08.688 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-03-03T06:10:08.688 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10425 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-03-03T06:10:08.688 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-03-03T06:10:08.688 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_dup_netdev.ko 2023-03-03T06:10:08.688 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_flow_table.ko 2023-03-03T06:10:08.688 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-03-03T06:10:08.688 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_log_syslog.ko 2023-03-03T06:10:08.688 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 91809 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_nat.ko 2023-03-03T06:10:08.689 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9625 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_nat_amanda.ko 2023-03-03T06:10:08.689 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_nat_ftp.ko 2023-03-03T06:10:08.689 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_nat_irc.ko 2023-03-03T06:10:08.689 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_nat_sip.ko 2023-03-03T06:10:08.689 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_nat_tftp.ko 2023-03-03T06:10:08.689 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_synproxy_core.ko 2023-03-03T06:10:08.689 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 644793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nf_tables.ko 2023-03-03T06:10:08.690 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50457 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nfnetlink.ko 2023-03-03T06:10:08.690 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nfnetlink_acct.ko 2023-03-03T06:10:08.690 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-03-03T06:10:08.690 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nfnetlink_log.ko 2023-03-03T06:10:08.690 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nfnetlink_osf.ko 2023-03-03T06:10:08.690 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nfnetlink_queue.ko 2023-03-03T06:10:08.690 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9961 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_chain_nat.ko 2023-03-03T06:10:08.690 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33001 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_compat.ko 2023-03-03T06:10:08.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40121 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_ct.ko 2023-03-03T06:10:08.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_dup_netdev.ko 2023-03-03T06:10:08.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_fib.ko 2023-03-03T06:10:08.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-03-03T06:10:08.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_hash.ko 2023-03-03T06:10:08.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19545 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_limit.ko 2023-03-03T06:10:08.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14905 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_log.ko 2023-03-03T06:10:08.691 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_masq.ko 2023-03-03T06:10:08.692 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13889 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_nat.ko 2023-03-03T06:10:08.692 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_numgen.ko 2023-03-03T06:10:08.692 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_osf.ko 2023-03-03T06:10:08.692 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_queue.ko 2023-03-03T06:10:08.692 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_redir.ko 2023-03-03T06:10:08.692 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_reject.ko 2023-03-03T06:10:08.692 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_reject_inet.ko 2023-03-03T06:10:08.692 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_socket.ko 2023-03-03T06:10:08.693 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_synproxy.ko 2023-03-03T06:10:08.693 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/nft_tproxy.ko 2023-03-03T06:10:08.693 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 105865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/x_tables.ko 2023-03-03T06:10:08.693 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11881 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_AUDIT.ko 2023-03-03T06:10:08.693 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-03-03T06:10:08.693 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-03-03T06:10:08.693 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-03-03T06:10:08.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_CT.ko 2023-03-03T06:10:08.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_DSCP.ko 2023-03-03T06:10:08.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_HL.ko 2023-03-03T06:10:08.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13265 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_HMARK.ko 2023-03-03T06:10:08.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-03-03T06:10:08.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_LED.ko 2023-03-03T06:10:08.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11057 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_LOG.ko 2023-03-03T06:10:08.694 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-03-03T06:10:08.695 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_NETMAP.ko 2023-03-03T06:10:08.695 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_NFLOG.ko 2023-03-03T06:10:08.695 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13161 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-03-03T06:10:08.695 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_RATEEST.ko 2023-03-03T06:10:08.695 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_REDIRECT.ko 2023-03-03T06:10:08.695 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_SECMARK.ko 2023-03-03T06:10:08.695 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_TCPMSS.ko 2023-03-03T06:10:08.695 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-03-03T06:10:08.696 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19801 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_TEE.ko 2023-03-03T06:10:08.696 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_TPROXY.ko 2023-03-03T06:10:08.696 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_TRACE.ko 2023-03-03T06:10:08.696 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_addrtype.ko 2023-03-03T06:10:08.696 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13977 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_bpf.ko 2023-03-03T06:10:08.696 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16745 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_cgroup.ko 2023-03-03T06:10:08.696 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11217 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_cluster.ko 2023-03-03T06:10:08.696 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7073 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_comment.ko 2023-03-03T06:10:08.697 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_connbytes.ko 2023-03-03T06:10:08.697 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11169 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_connlabel.ko 2023-03-03T06:10:08.697 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_connlimit.ko 2023-03-03T06:10:08.697 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_connmark.ko 2023-03-03T06:10:08.697 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_conntrack.ko 2023-03-03T06:10:08.697 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_cpu.ko 2023-03-03T06:10:08.697 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_dccp.ko 2023-03-03T06:10:08.698 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_devgroup.ko 2023-03-03T06:10:08.698 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_dscp.ko 2023-03-03T06:10:08.698 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_ecn.ko 2023-03-03T06:10:08.698 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_esp.ko 2023-03-03T06:10:08.698 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50401 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_hashlimit.ko 2023-03-03T06:10:08.698 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_helper.ko 2023-03-03T06:10:08.698 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_hl.ko 2023-03-03T06:10:08.698 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_ipcomp.ko 2023-03-03T06:10:08.699 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_iprange.ko 2023-03-03T06:10:08.699 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_ipvs.ko 2023-03-03T06:10:08.699 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_l2tp.ko 2023-03-03T06:10:08.699 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7505 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_length.ko 2023-03-03T06:10:08.699 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_limit.ko 2023-03-03T06:10:08.699 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_mac.ko 2023-03-03T06:10:08.699 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_mark.ko 2023-03-03T06:10:08.699 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_multiport.ko 2023-03-03T06:10:08.700 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_nat.ko 2023-03-03T06:10:08.700 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9969 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_nfacct.ko 2023-03-03T06:10:08.700 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7865 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_osf.ko 2023-03-03T06:10:08.700 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9081 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_owner.ko 2023-03-03T06:10:08.700 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9585 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_physdev.ko 2023-03-03T06:10:08.700 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_pkttype.ko 2023-03-03T06:10:08.700 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11713 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_policy.ko 2023-03-03T06:10:08.700 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9049 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_quota.ko 2023-03-03T06:10:08.701 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_rateest.ko 2023-03-03T06:10:08.701 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_realm.ko 2023-03-03T06:10:08.701 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41633 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_recent.ko 2023-03-03T06:10:08.701 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_sctp.ko 2023-03-03T06:10:08.701 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_set.ko 2023-03-03T06:10:08.701 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16089 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_socket.ko 2023-03-03T06:10:08.701 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_state.ko 2023-03-03T06:10:08.702 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8985 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_statistic.ko 2023-03-03T06:10:08.702 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8313 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_string.ko 2023-03-03T06:10:08.702 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_tcpmss.ko 2023-03-03T06:10:08.702 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_tcpudp.ko 2023-03-03T06:10:08.702 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10833 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_time.ko 2023-03-03T06:10:08.702 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8617 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netfilter/xt_u32.ko 2023-03-03T06:10:08.702 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netlink/ 2023-03-03T06:10:08.702 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13417 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/netlink/netlink_diag.ko 2023-03-03T06:10:08.703 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/nsh/ 2023-03-03T06:10:08.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/nsh/nsh.ko 2023-03-03T06:10:08.703 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/openvswitch/ 2023-03-03T06:10:08.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 400273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/openvswitch/openvswitch.ko 2023-03-03T06:10:08.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/openvswitch/vport-geneve.ko 2023-03-03T06:10:08.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8953 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/openvswitch/vport-gre.ko 2023-03-03T06:10:08.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11297 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/openvswitch/vport-vxlan.ko 2023-03-03T06:10:08.703 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/packet/ 2023-03-03T06:10:08.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12561 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/packet/af_packet_diag.ko 2023-03-03T06:10:08.704 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/phonet/ 2023-03-03T06:10:08.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124025 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/phonet/phonet.ko 2023-03-03T06:10:08.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/phonet/pn_pep.ko 2023-03-03T06:10:08.704 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/rds/ 2023-03-03T06:10:08.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 251929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/rds/rds.ko 2023-03-03T06:10:08.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 172945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/rds/rds_rdma.ko 2023-03-03T06:10:08.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70913 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/rds/rds_tcp.ko 2023-03-03T06:10:08.705 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/rxrpc/ 2023-03-03T06:10:08.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 781177 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/rxrpc/rxrpc.ko 2023-03-03T06:10:08.705 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/ 2023-03-03T06:10:08.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33473 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_csum.ko 2023-03-03T06:10:08.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25305 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_gact.ko 2023-03-03T06:10:08.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_ipt.ko 2023-03-03T06:10:08.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_mirred.ko 2023-03-03T06:10:08.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22777 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_nat.ko 2023-03-03T06:10:08.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_pedit.ko 2023-03-03T06:10:08.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33185 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_police.ko 2023-03-03T06:10:08.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21681 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_simple.ko 2023-03-03T06:10:08.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_skbedit.ko 2023-03-03T06:10:08.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37825 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/act_vlan.ko 2023-03-03T06:10:08.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24937 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_basic.ko 2023-03-03T06:10:08.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32569 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_bpf.ko 2023-03-03T06:10:08.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_cgroup.ko 2023-03-03T06:10:08.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_flow.ko 2023-03-03T06:10:08.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28993 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_fw.ko 2023-03-03T06:10:08.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35577 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_route.ko 2023-03-03T06:10:08.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_rsvp.ko 2023-03-03T06:10:08.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35033 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_rsvp6.ko 2023-03-03T06:10:08.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42041 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_tcindex.ko 2023-03-03T06:10:08.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56441 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/cls_u32.ko 2023-03-03T06:10:08.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6945 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/em_cmp.ko 2023-03-03T06:10:08.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12857 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/em_ipset.ko 2023-03-03T06:10:08.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/em_meta.ko 2023-03-03T06:10:08.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/em_nbyte.ko 2023-03-03T06:10:08.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10129 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/em_text.ko 2023-03-03T06:10:08.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/em_u32.ko 2023-03-03T06:10:08.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43737 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_cbq.ko 2023-03-03T06:10:08.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_choke.ko 2023-03-03T06:10:08.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19201 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_codel.ko 2023-03-03T06:10:08.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_drr.ko 2023-03-03T06:10:08.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32753 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_dsmark.ko 2023-03-03T06:10:08.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30153 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_fq.ko 2023-03-03T06:10:08.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28929 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_fq_codel.ko 2023-03-03T06:10:08.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32729 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_gred.ko 2023-03-03T06:10:08.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44273 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_hfsc.ko 2023-03-03T06:10:08.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20369 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_hhf.ko 2023-03-03T06:10:08.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63257 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_htb.ko 2023-03-03T06:10:08.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15137 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_ingress.ko 2023-03-03T06:10:08.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22113 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_mqprio.ko 2023-03-03T06:10:08.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_multiq.ko 2023-03-03T06:10:08.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33649 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_netem.ko 2023-03-03T06:10:08.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23497 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_pie.ko 2023-03-03T06:10:08.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_plug.ko 2023-03-03T06:10:08.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_prio.ko 2023-03-03T06:10:08.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44409 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_qfq.ko 2023-03-03T06:10:08.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28609 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_red.ko 2023-03-03T06:10:08.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24769 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_sfb.ko 2023-03-03T06:10:08.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27897 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_sfq.ko 2023-03-03T06:10:08.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_tbf.ko 2023-03-03T06:10:08.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26641 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sched/sch_teql.ko 2023-03-03T06:10:08.711 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sctp/ 2023-03-03T06:10:08.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 921329 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sctp/sctp.ko 2023-03-03T06:10:08.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18537 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sctp/sctp_diag.ko 2023-03-03T06:10:08.712 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/ 2023-03-03T06:10:08.712 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/auth_gss/ 2023-03-03T06:10:08.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 339241 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-03-03T06:10:08.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63193 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-03-03T06:10:08.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1476289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/sunrpc.ko 2023-03-03T06:10:08.713 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/xprtrdma/ 2023-03-03T06:10:08.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 720761 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-03-03T06:10:08.713 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/tipc/ 2023-03-03T06:10:08.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9529 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/tipc/diag.ko 2023-03-03T06:10:08.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1123321 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/tipc/tipc.ko 2023-03-03T06:10:08.713 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/unix/ 2023-03-03T06:10:08.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15233 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/unix/unix_diag.ko 2023-03-03T06:10:08.713 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/vmw_vsock/ 2023-03-03T06:10:08.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77785 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-03-03T06:10:08.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68289 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-03-03T06:10:08.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/vmw_vsock/vsock.ko 2023-03-03T06:10:08.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10209 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/vmw_vsock/vsock_diag.ko 2023-03-03T06:10:08.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18393 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-03-03T06:10:08.714 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/x25/ 2023-03-03T06:10:08.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125817 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/x25/x25.ko 2023-03-03T06:10:08.714 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/xfrm/ 2023-03-03T06:10:08.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20793 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/xfrm/xfrm_algo.ko 2023-03-03T06:10:08.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20009 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-03-03T06:10:08.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101665 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/net/xfrm/xfrm_user.ko 2023-03-03T06:10:08.715 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/virt/ 2023-03-03T06:10:08.715 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/virt/lib/ 2023-03-03T06:10:08.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14433 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/kernel/virt/lib/irqbypass.ko 2023-03-03T06:10:08.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 633131 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.alias 2023-03-03T06:10:08.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 619504 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.alias.bin 2023-03-03T06:10:08.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8377 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.builtin 2023-03-03T06:10:08.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22490 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.builtin.alias.bin 2023-03-03T06:10:08.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10648 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.builtin.bin 2023-03-03T06:10:08.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69838 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.builtin.modinfo 2023-03-03T06:10:08.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 147260 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.dep 2023-03-03T06:10:08.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 234530 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.dep.bin 2023-03-03T06:10:08.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 210 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.devname 2023-03-03T06:10:08.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73341 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.order 2023-03-03T06:10:08.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 830 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.softdep 2023-03-03T06:10:08.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 250429 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.symbols 2023-03-03T06:10:08.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 304835 2023-03-03 04:49 ./lib/modules/6.2.0-ceph-g75534f74ac3d/modules.symbols.bin 2023-03-03T06:10:08.717 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./usr/ 2023-03-03T06:10:08.717 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./usr/share/ 2023-03-03T06:10:08.717 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./usr/share/doc/ 2023-03-03T06:10:08.717 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-03-03 04:49 ./usr/share/doc/linux-image-6.2.0-ceph-g75534f74ac3d/ 2023-03-03T06:10:08.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 176 2023-03-03 04:49 ./usr/share/doc/linux-image-6.2.0-ceph-g75534f74ac3d/changelog.Debian.gz 2023-03-03T06:10:08.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 692 2023-03-03 04:49 ./usr/share/doc/linux-image-6.2.0-ceph-g75534f74ac3d/copyright 2023-03-03T06:10:08.718 DEBUG:teuthology.task.kernel:get_image_version: 6.2.0-ceph-g75534f74ac3d 2023-03-03T06:10:08.719 INFO:teuthology.task.kernel:searching for kernel 6.2.0-ceph-g75534f74ac3d 2023-03-03T06:10:08.719 DEBUG:teuthology.orchestra.run.smithi032:> egrep '(submenu|menuentry.*6.2.0-ceph-g75534f74ac3d).*{' /boot/grub/grub.cfg 2023-03-03T06:10:08.728 INFO:teuthology.orchestra.run.smithi032.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-03-03T06:10:08.728 INFO:teuthology.orchestra.run.smithi032.stdout: menuentry 'Ubuntu, with Linux 6.2.0-ceph-g75534f74ac3d' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.2.0-ceph-g75534f74ac3d-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-03-03T06:10:08.728 INFO:teuthology.orchestra.run.smithi032.stdout: menuentry 'Ubuntu, with Linux 6.2.0-ceph-g75534f74ac3d (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.2.0-ceph-g75534f74ac3d-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-03-03T06:10:08.729 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-03-03T06:10:08.729 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.2.0-ceph-g75534f74ac3d 2023-03-03T06:10:08.729 DEBUG:teuthology.orchestra.run.smithi032:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-03-03T06:10:08.731 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi032.front.sepia.ceph.com to complete... 2023-03-03T06:10:09.528 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub' 2023-03-03T06:10:09.534 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-03-03T06:10:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2023-03-03T06:10:09.844 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:10:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-6.2.0-ceph-g75534f74ac3d 2023-03-03T06:10:10.236 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-03-03T06:10:10.248 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-03-03T06:10:11.869 INFO:teuthology.orchestra.run.smithi032.stderr:done 2023-03-03T06:10:20.029 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-03-03T06:10:20.029 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-03-03T06:10:20.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-03T06:10:38.469 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-03-03T06:10:42.989 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-03-03T06:10:42.989 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-03-03T06:10:42.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-03T06:11:01.509 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-03-03T06:11:08.477 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-03-03T06:11:08.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-03T06:11:11.561 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-03-03T06:11:31.529 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-03-03T06:11:31.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-03T06:11:41.572 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-03-03T06:11:41.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-03T06:11:41.826 DEBUG:teuthology.orchestra.run.smithi003:> true 2023-03-03T06:11:42.589 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-03-03T06:11:42.589 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.2.0-ceph-g75534f74ac3d"... 2023-03-03T06:11:42.590 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2023-03-03T06:11:42.659 INFO:teuthology.orchestra.run.smithi003.stdout:6.2.0-ceph-g75534f74ac3d 2023-03-03T06:11:42.660 DEBUG:teuthology.task.kernel:current kernel version is 6.2.0-ceph-g75534f74ac3d vs 6.2.0-ceph-g75534f74ac3d 2023-03-03T06:11:42.660 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-03-03T06:11:42.660 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-03-03T06:11:43.662 DEBUG:teuthology.parallel:result is None 2023-03-03T06:11:47.220 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-03-03T06:11:48.337 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-03-03T06:11:48.338 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.2.0-ceph-g75534f74ac3d"... 2023-03-03T06:11:48.338 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-03-03T06:11:48.356 INFO:teuthology.orchestra.run.smithi032.stdout:6.2.0-ceph-g75534f74ac3d 2023-03-03T06:11:48.356 DEBUG:teuthology.task.kernel:current kernel version is 6.2.0-ceph-g75534f74ac3d vs 6.2.0-ceph-g75534f74ac3d 2023-03-03T06:11:48.356 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-03-03T06:11:48.356 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-03-03T06:11:49.358 DEBUG:teuthology.parallel:result is None 2023-03-03T06:11:49.358 INFO:teuthology.run_tasks:Running task internal.base... 2023-03-03T06:11:49.465 INFO:teuthology.task.internal:Creating test directory... 2023-03-03T06:11:49.466 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-03T06:11:49.475 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-03T06:11:49.496 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-03-03T06:11:49.594 INFO:teuthology.run_tasks:Running task internal.archive... 2023-03-03T06:11:49.609 INFO:teuthology.task.internal:Creating archive directory... 2023-03-03T06:11:49.610 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-03T06:11:49.618 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-03T06:11:49.669 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-03-03T06:11:49.681 INFO:teuthology.task.internal:Enabling coredump saving... 2023-03-03T06:11:49.682 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-03T06:11:49.694 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-03T06:11:49.801 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-03T06:11:49.825 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-03T06:11:49.836 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-03T06:11:49.868 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-03T06:11:49.875 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-03-03T06:11:49.916 INFO:teuthology.task.internal:Configuring sudo... 2023-03-03T06:11:49.916 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-03T06:11:49.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-03T06:11:49.968 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-03-03T06:11:49.983 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-03-03T06:11:49.984 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-03T06:11:50.018 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-03T06:11:50.039 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-03T06:11:50.097 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-03T06:11:50.164 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:11:50.164 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-03T06:11:50.255 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-03T06:11:50.280 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-03T06:11:50.344 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:11:50.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-03T06:11:50.455 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2023-03-03T06:11:50.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2023-03-03T06:11:50.745 INFO:teuthology.run_tasks:Running task internal.timer... 2023-03-03T06:11:50.784 INFO:teuthology.task.internal:Starting timer... 2023-03-03T06:11:50.785 INFO:teuthology.run_tasks:Running task pcp... 2023-03-03T06:11:50.820 INFO:teuthology.run_tasks:Running task selinux... 2023-03-03T06:11:50.845 DEBUG:teuthology.task.selinux:Excluding smithi003: OS 'ubuntu' does not support SELinux 2023-03-03T06:11:50.846 DEBUG:teuthology.task.selinux:Excluding smithi032: OS 'ubuntu' does not support SELinux 2023-03-03T06:11:50.846 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-03-03T06:11:50.846 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-03-03T06:11:50.846 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-03-03T06:11:50.847 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-03-03T06:11:50.856 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-03-03T06:11:50.856 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-03-03T06:11:50.881 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-03-03T06:11:50.883 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi032.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-03-03T06:20:02.640 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi032.front.sepia.ceph.com')] 2023-03-03T06:20:02.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-03-03T06:20:02.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-03T06:20:02.755 DEBUG:teuthology.orchestra.run.smithi003:> true 2023-03-03T06:20:03.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-03-03T06:20:03.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-03T06:20:03.148 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-03-03T06:20:03.432 INFO:teuthology.run_tasks:Running task clock... 2023-03-03T06:20:03.515 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-03-03T06:20:03.516 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-03T06:20:03.517 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-03T06:20:03.525 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-03T06:20:03.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-03T06:20:03.615 INFO:teuthology.orchestra.run.smithi003.stdout: 3 Mar 06:20:03 ntpd[19939]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-03-03T06:20:03.615 INFO:teuthology.orchestra.run.smithi003.stdout: 3 Mar 06:20:03 ntpd[19939]: Command line: ntpd -gq 2023-03-03T06:20:03.616 INFO:teuthology.orchestra.run.smithi003.stdout: 3 Mar 06:20:03 ntpd[19939]: proto: precision = 0.043 usec (-24) 2023-03-03T06:20:03.617 INFO:teuthology.orchestra.run.smithi003.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-03-03T06:20:03.617 INFO:teuthology.orchestra.run.smithi003.stderr:restrict ::: KOD does nothing without LIMITED. 2023-03-03T06:20:03.618 INFO:teuthology.orchestra.run.smithi003.stdout: 3 Mar 06:20:03 ntpd[19939]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-03-03T06:20:03.618 INFO:teuthology.orchestra.run.smithi003.stdout: 3 Mar 06:20:03 ntpd[19939]: restrict ::: KOD does nothing without LIMITED. 2023-03-03T06:20:03.619 INFO:teuthology.orchestra.run.smithi003.stdout: 3 Mar 06:20:03 ntpd[19939]: Listen and drop on 0 v6wildcard [::]:123 2023-03-03T06:20:03.619 INFO:teuthology.orchestra.run.smithi003.stdout: 3 Mar 06:20:03 ntpd[19939]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-03-03T06:20:03.619 INFO:teuthology.orchestra.run.smithi003.stdout: 3 Mar 06:20:03 ntpd[19939]: Listen normally on 2 lo 127.0.0.1:123 2023-03-03T06:20:03.619 INFO:teuthology.orchestra.run.smithi003.stdout: 3 Mar 06:20:03 ntpd[19939]: Listen normally on 3 ens1f0 172.21.15.3:123 2023-03-03T06:20:03.620 INFO:teuthology.orchestra.run.smithi003.stdout: 3 Mar 06:20:03 ntpd[19939]: Listen normally on 4 lo [::1]:123 2023-03-03T06:20:03.620 INFO:teuthology.orchestra.run.smithi003.stdout: 3 Mar 06:20:03 ntpd[19939]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15dc%3]:123 2023-03-03T06:20:03.620 INFO:teuthology.orchestra.run.smithi003.stdout: 3 Mar 06:20:03 ntpd[19939]: Listening on routing socket on fd #22 for interface updates 2023-03-03T06:20:03.625 INFO:teuthology.orchestra.run.smithi032.stdout: 3 Mar 06:20:03 ntpd[19871]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-03-03T06:20:03.626 INFO:teuthology.orchestra.run.smithi032.stdout: 3 Mar 06:20:03 ntpd[19871]: Command line: ntpd -gq 2023-03-03T06:20:03.627 INFO:teuthology.orchestra.run.smithi032.stdout: 3 Mar 06:20:03 ntpd[19871]: proto: precision = 0.047 usec (-24) 2023-03-03T06:20:03.627 INFO:teuthology.orchestra.run.smithi032.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-03-03T06:20:03.628 INFO:teuthology.orchestra.run.smithi032.stdout: 3 Mar 06:20:03 ntpd[19871]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-03-03T06:20:03.628 INFO:teuthology.orchestra.run.smithi032.stdout: 3 Mar 06:20:03 ntpd[19871]: restrict ::: KOD does nothing without LIMITED. 2023-03-03T06:20:03.628 INFO:teuthology.orchestra.run.smithi032.stderr:restrict ::: KOD does nothing without LIMITED. 2023-03-03T06:20:03.629 INFO:teuthology.orchestra.run.smithi032.stdout: 3 Mar 06:20:03 ntpd[19871]: Listen and drop on 0 v6wildcard [::]:123 2023-03-03T06:20:03.629 INFO:teuthology.orchestra.run.smithi032.stdout: 3 Mar 06:20:03 ntpd[19871]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-03-03T06:20:03.629 INFO:teuthology.orchestra.run.smithi032.stdout: 3 Mar 06:20:03 ntpd[19871]: Listen normally on 2 lo 127.0.0.1:123 2023-03-03T06:20:03.630 INFO:teuthology.orchestra.run.smithi032.stdout: 3 Mar 06:20:03 ntpd[19871]: Listen normally on 3 ens1f0 172.21.15.32:123 2023-03-03T06:20:03.630 INFO:teuthology.orchestra.run.smithi032.stdout: 3 Mar 06:20:03 ntpd[19871]: Listen normally on 4 lo [::1]:123 2023-03-03T06:20:03.630 INFO:teuthology.orchestra.run.smithi032.stdout: 3 Mar 06:20:03 ntpd[19871]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:2602%2]:123 2023-03-03T06:20:03.630 INFO:teuthology.orchestra.run.smithi032.stdout: 3 Mar 06:20:03 ntpd[19871]: Listening on routing socket on fd #22 for interface updates 2023-03-03T06:20:04.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:04.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:04.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:04.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:04.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:04.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:04.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:04.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:04.618 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:04.618 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:04.618 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:04.619 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:04.619 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:04.619 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:04 ntpd[19939]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:04.626 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:04 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:04.626 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:04 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:04.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:04 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:04.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:04 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:04.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:04 ntpd[19871]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:04.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:04 ntpd[19871]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:05.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:05 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:05.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:05 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:05.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:05 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:05.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:05 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:05.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:05 ntpd[19939]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:05.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:05 ntpd[19939]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:05.626 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:05 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:05.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:05 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:05.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:05 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:05.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:05 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:05.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:05 ntpd[19871]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:05.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:05 ntpd[19871]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:06.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:06 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:06.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:06 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:06.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:06 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:06.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:06 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:06.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:06 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:06.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:06 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:06.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:06 ntpd[19939]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:06.618 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:06 ntpd[19939]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:06.618 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:06 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:06.618 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:06 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:06.618 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:06 ntpd[19939]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:06.618 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:06 ntpd[19939]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:06.626 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:06 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:06.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:06 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:06.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:06 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:06.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:06 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:06.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:06 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:06.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:06 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:06.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:06 ntpd[19871]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:06.629 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:06 ntpd[19871]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:07.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:07 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:07.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:07 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:07.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:07 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:07.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:07 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:07.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:07 ntpd[19939]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:07.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:07 ntpd[19939]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:07.626 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:07 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:07.626 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:07 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:07.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:07 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:07.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:07 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:07.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:07 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:07.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:07 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:07.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:07 ntpd[19871]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:07.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:07 ntpd[19871]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:07.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:07 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:07.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:07 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:07.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:07 ntpd[19871]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:07.629 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:07 ntpd[19871]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:08.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:08 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:08.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:08 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:08.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:08 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:08.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:08 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:08.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:08 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:08.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:08 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:08.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:08 ntpd[19939]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:08.618 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:08 ntpd[19939]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:08.618 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:08 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:08.618 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:08 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:08.618 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:08 ntpd[19939]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:08.619 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:08 ntpd[19939]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:08.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:08 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:08.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:08 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:08.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:08 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:08.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:08 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:08.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:08 ntpd[19871]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:08.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:08 ntpd[19871]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:09.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:09 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:09.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:09 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:09.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:09 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:09.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:09 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:09.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:09 ntpd[19939]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:09.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:09 ntpd[19939]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:09.626 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:09 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:09.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:09 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:09.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:09 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:09.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:09 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:09.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:09 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:09.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:09 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:09.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:09 ntpd[19871]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:09.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:09 ntpd[19871]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:09.629 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:09 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:09.629 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:09 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:09.629 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:09 ntpd[19871]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:09.629 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:09 ntpd[19871]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:10.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:10 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:10.616 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:10 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:10.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:10 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:10.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:10 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:10.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:10 ntpd[19939]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:10.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:10 ntpd[19939]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:10.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:10 ntpd[19939]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:10.617 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:10 ntpd[19939]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:10.618 INFO:teuthology.orchestra.run.smithi003.stdout: 3 Mar 06:20:10 ntpd[19939]: ntpd: time slew +0.001306 s 2023-03-03T06:20:10.619 INFO:teuthology.orchestra.run.smithi003.stdout:ntpd: time slew +0.001306s 2023-03-03T06:20:10.619 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:10 ntpd[19939]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-03-03T06:20:10.619 INFO:teuthology.orchestra.run.smithi003.stderr: 3 Mar 06:20:10 ntpd[19939]: can't open /var/log/ntpstats/loopstats.20230303: Permission denied 2023-03-03T06:20:10.626 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:10 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:10.626 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:10 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:10.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:10 ntpd[19871]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-03-03T06:20:10.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:10 ntpd[19871]: can't open /var/log/ntpstats/rawstats.20230303: Permission denied 2023-03-03T06:20:10.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:10 ntpd[19871]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-03-03T06:20:10.627 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:10 ntpd[19871]: can't open /var/log/ntpstats/peerstats.20230303: Permission denied 2023-03-03T06:20:10.628 INFO:teuthology.orchestra.run.smithi032.stdout: 3 Mar 06:20:10 ntpd[19871]: ntpd: time slew -0.000039 s 2023-03-03T06:20:10.628 INFO:teuthology.orchestra.run.smithi032.stdout:ntpd: time slew -0.000039s 2023-03-03T06:20:10.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:10 ntpd[19871]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-03-03T06:20:10.628 INFO:teuthology.orchestra.run.smithi032.stderr: 3 Mar 06:20:10 ntpd[19871]: can't open /var/log/ntpstats/loopstats.20230303: Permission denied 2023-03-03T06:20:10.780 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2023-03-03T06:20:10.780 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2023-03-03T06:20:10.780 INFO:teuthology.orchestra.run.smithi032.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-03-03T06:20:10.781 INFO:teuthology.orchestra.run.smithi032.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-03-03T06:20:10.781 INFO:teuthology.orchestra.run.smithi032.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-03-03T06:20:10.781 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-03-03T06:20:10.787 INFO:teuthology.orchestra.run.smithi003.stdout: remote refid st t when poll reach delay offset jitter 2023-03-03T06:20:10.787 INFO:teuthology.orchestra.run.smithi003.stdout:============================================================================== 2023-03-03T06:20:10.787 INFO:teuthology.orchestra.run.smithi003.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-03-03T06:20:10.788 INFO:teuthology.orchestra.run.smithi003.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-03-03T06:20:10.788 INFO:teuthology.orchestra.run.smithi003.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-03-03T06:20:10.788 INFO:teuthology.orchestra.run.smithi003.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-03-03T06:20:10.791 INFO:teuthology.run_tasks:Running task install... 2023-03-03T06:20:10.801 DEBUG:teuthology.task.install:project ceph 2023-03-03T06:20:10.802 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '472aaef4ec3b3a2c29f73ca67854785a910f1f9b'}} 2023-03-03T06:20:10.802 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '472aaef4ec3b3a2c29f73ca67854785a910f1f9b'} 2023-03-03T06:20:10.802 INFO:teuthology.task.install:Using flavor: default 2023-03-03T06:20:10.808 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-03-03T06:20:10.808 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-03-03T06:20:10.809 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-key list | grep Ceph 2023-03-03T06:20:10.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-key list | grep Ceph 2023-03-03T06:20:11.123 INFO:teuthology.orchestra.run.smithi003.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-03-03T06:20:11.124 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-03-03T06:20:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-03-03T06:20:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph.com (release key) 2023-03-03T06:20:11.333 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-03-03T06:20:11.333 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-03-03T06:20:11.333 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=472aaef4ec3b3a2c29f73ca67854785a910f1f9b 2023-03-03T06:20:11.344 INFO:teuthology.orchestra.run.smithi003.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-03-03T06:20:11.345 INFO:teuthology.orchestra.run.smithi003.stdout:uid [ unknown] Ceph.com (release key) 2023-03-03T06:20:11.348 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-03-03T06:20:11.348 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-03-03T06:20:11.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=472aaef4ec3b3a2c29f73ca67854785a910f1f9b 2023-03-03T06:20:11.480 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default/ 2023-03-03T06:20:11.480 INFO:teuthology.task.install.deb:Package version is 18.0.0-2739-g472aaef4-1focal 2023-03-03T06:20:11.497 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default/ 2023-03-03T06:20:11.498 INFO:teuthology.task.install.deb:Package version is 18.0.0-2739-g472aaef4-1focal 2023-03-03T06:20:11.600 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:20:11.601 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-03-03T06:20:11.605 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:20:11.606 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-03-03T06:20:11.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2023-03-03T06:20:11.652 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-get update 2023-03-03T06:20:11.972 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-03-03T06:20:11.973 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-03-03T06:20:11.973 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:3 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-03-03T06:20:11.977 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-03-03T06:20:12.007 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal InRelease 2023-03-03T06:20:12.017 INFO:teuthology.orchestra.run.smithi003.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal InRelease 2023-03-03T06:20:12.030 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-03-03T06:20:12.039 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-03-03T06:20:12.055 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal Release.gpg 2023-03-03T06:20:12.063 INFO:teuthology.orchestra.run.smithi003.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal Release.gpg 2023-03-03T06:20:12.085 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 Packages [17.3 kB] 2023-03-03T06:20:12.095 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal InRelease 2023-03-03T06:20:12.095 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:5 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-03-03T06:20:12.095 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main i386 Packages [3,423 B] 2023-03-03T06:20:12.137 INFO:teuthology.orchestra.run.smithi003.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main i386 Packages [3,423 B] 2023-03-03T06:20:12.140 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:7 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-03-03T06:20:12.145 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 Packages [17.3 kB] 2023-03-03T06:20:12.221 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-03-03T06:20:13.956 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 28.4 kB in 2s (15.1 kB/s) 2023-03-03T06:20:14.074 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 28.4 kB in 2s (14.1 kB/s) 2023-03-03T06:20:15.269 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2023-03-03T06:20:15.326 DEBUG:teuthology.orchestra.run.smithi003:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-2739-g472aaef4-1focal cephadm=18.0.0-2739-g472aaef4-1focal ceph-mds=18.0.0-2739-g472aaef4-1focal ceph-mgr=18.0.0-2739-g472aaef4-1focal ceph-common=18.0.0-2739-g472aaef4-1focal ceph-fuse=18.0.0-2739-g472aaef4-1focal ceph-test=18.0.0-2739-g472aaef4-1focal ceph-volume=18.0.0-2739-g472aaef4-1focal radosgw=18.0.0-2739-g472aaef4-1focal python3-rados=18.0.0-2739-g472aaef4-1focal python3-rgw=18.0.0-2739-g472aaef4-1focal python3-cephfs=18.0.0-2739-g472aaef4-1focal python3-rbd=18.0.0-2739-g472aaef4-1focal libcephfs2=18.0.0-2739-g472aaef4-1focal libcephfs-dev=18.0.0-2739-g472aaef4-1focal librados2=18.0.0-2739-g472aaef4-1focal librbd1=18.0.0-2739-g472aaef4-1focal rbd-fuse=18.0.0-2739-g472aaef4-1focal python3-cephfs=18.0.0-2739-g472aaef4-1focal cephfs-shell=18.0.0-2739-g472aaef4-1focal cephfs-top=18.0.0-2739-g472aaef4-1focal cephfs-mirror=18.0.0-2739-g472aaef4-1focal 2023-03-03T06:20:15.394 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2023-03-03T06:20:15.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-2739-g472aaef4-1focal cephadm=18.0.0-2739-g472aaef4-1focal ceph-mds=18.0.0-2739-g472aaef4-1focal ceph-mgr=18.0.0-2739-g472aaef4-1focal ceph-common=18.0.0-2739-g472aaef4-1focal ceph-fuse=18.0.0-2739-g472aaef4-1focal ceph-test=18.0.0-2739-g472aaef4-1focal ceph-volume=18.0.0-2739-g472aaef4-1focal radosgw=18.0.0-2739-g472aaef4-1focal python3-rados=18.0.0-2739-g472aaef4-1focal python3-rgw=18.0.0-2739-g472aaef4-1focal python3-cephfs=18.0.0-2739-g472aaef4-1focal python3-rbd=18.0.0-2739-g472aaef4-1focal libcephfs2=18.0.0-2739-g472aaef4-1focal libcephfs-dev=18.0.0-2739-g472aaef4-1focal librados2=18.0.0-2739-g472aaef4-1focal librbd1=18.0.0-2739-g472aaef4-1focal rbd-fuse=18.0.0-2739-g472aaef4-1focal python3-cephfs=18.0.0-2739-g472aaef4-1focal cephfs-shell=18.0.0-2739-g472aaef4-1focal cephfs-top=18.0.0-2739-g472aaef4-1focal cephfs-mirror=18.0.0-2739-g472aaef4-1focal 2023-03-03T06:20:15.472 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2023-03-03T06:20:15.600 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2023-03-03T06:20:15.675 INFO:teuthology.orchestra.run.smithi003.stdout:Building dependency tree... 2023-03-03T06:20:15.675 INFO:teuthology.orchestra.run.smithi003.stdout:Reading state information... 2023-03-03T06:20:15.796 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2023-03-03T06:20:15.796 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2023-03-03T06:20:15.867 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages were automatically installed and are no longer required: 2023-03-03T06:20:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-03-03T06:20:15.868 INFO:teuthology.orchestra.run.smithi003.stdout:Use 'sudo apt autoremove' to remove them. 2023-03-03T06:20:15.868 INFO:teuthology.orchestra.run.smithi003.stdout:The following additional packages will be installed: 2023-03-03T06:20:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-03-03T06:20:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-03-03T06:20:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-03-03T06:20:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-03-03T06:20:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-03-03T06:20:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-03-03T06:20:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-03-03T06:20:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-03-03T06:20:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-03-03T06:20:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2023-03-03T06:20:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-03-03T06:20:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-03-03T06:20:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-03-03T06:20:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-03-03T06:20:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-03-03T06:20:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-03-03T06:20:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2023-03-03T06:20:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2023-03-03T06:20:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-03-03T06:20:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-03-03T06:20:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-03-03T06:20:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-03-03T06:20:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: qttranslations5-l10n socat xmlstarlet 2023-03-03T06:20:15.874 INFO:teuthology.orchestra.run.smithi003.stdout:Suggested packages: 2023-03-03T06:20:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-03-03T06:20:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-03-03T06:20:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-03-03T06:20:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-03-03T06:20:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-03-03T06:20:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-03-03T06:20:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-03-03T06:20:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-03-03T06:20:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-03-03T06:20:15.875 INFO:teuthology.orchestra.run.smithi003.stdout:Recommended packages: 2023-03-03T06:20:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: btrfs-tools 2023-03-03T06:20:15.983 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2023-03-03T06:20:15.983 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-03-03T06:20:15.984 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2023-03-03T06:20:15.984 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2023-03-03T06:20:15.984 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-03-03T06:20:15.984 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-03-03T06:20:15.985 INFO:teuthology.orchestra.run.smithi032.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-03-03T06:20:15.985 INFO:teuthology.orchestra.run.smithi032.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-03-03T06:20:15.985 INFO:teuthology.orchestra.run.smithi032.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-03-03T06:20:15.985 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-03-03T06:20:15.986 INFO:teuthology.orchestra.run.smithi032.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-03-03T06:20:15.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-03-03T06:20:15.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-03-03T06:20:15.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2023-03-03T06:20:15.986 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-03-03T06:20:15.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-03-03T06:20:15.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-03-03T06:20:15.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-03-03T06:20:15.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-03-03T06:20:15.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-03-03T06:20:15.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2023-03-03T06:20:15.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2023-03-03T06:20:15.987 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-03-03T06:20:15.988 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-03-03T06:20:15.988 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-03-03T06:20:15.988 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-03-03T06:20:15.988 INFO:teuthology.orchestra.run.smithi032.stdout: qttranslations5-l10n socat xmlstarlet 2023-03-03T06:20:15.988 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2023-03-03T06:20:15.989 INFO:teuthology.orchestra.run.smithi032.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-03-03T06:20:15.989 INFO:teuthology.orchestra.run.smithi032.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-03-03T06:20:15.989 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-03-03T06:20:15.989 INFO:teuthology.orchestra.run.smithi032.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-03-03T06:20:15.989 INFO:teuthology.orchestra.run.smithi032.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-03-03T06:20:15.989 INFO:teuthology.orchestra.run.smithi032.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-03-03T06:20:15.990 INFO:teuthology.orchestra.run.smithi032.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-03-03T06:20:15.990 INFO:teuthology.orchestra.run.smithi032.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-03-03T06:20:15.990 INFO:teuthology.orchestra.run.smithi032.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-03-03T06:20:15.990 INFO:teuthology.orchestra.run.smithi032.stdout:Recommended packages: 2023-03-03T06:20:15.990 INFO:teuthology.orchestra.run.smithi032.stdout: btrfs-tools 2023-03-03T06:20:16.129 INFO:teuthology.orchestra.run.smithi003.stdout:The following NEW packages will be installed: 2023-03-03T06:20:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-03-03T06:20:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-03-03T06:20:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-03-03T06:20:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-03-03T06:20:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-03-03T06:20:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-03-03T06:20:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-03-03T06:20:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-03-03T06:20:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-03-03T06:20:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-03-03T06:20:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-03-03T06:20:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2023-03-03T06:20:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-03-03T06:20:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-03-03T06:20:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-03-03T06:20:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-03-03T06:20:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-03-03T06:20:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-03-03T06:20:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-03-03T06:20:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-03-03T06:20:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-03-03T06:20:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-03-03T06:20:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-03-03T06:20:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-03-03T06:20:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-03-03T06:20:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-03-03T06:20:16.134 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages will be upgraded: 2023-03-03T06:20:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 librbd1 2023-03-03T06:20:16.235 INFO:teuthology.orchestra.run.smithi003.stdout:2 upgraded, 116 newly installed, 0 to remove and 152 not upgraded. 2023-03-03T06:20:16.236 INFO:teuthology.orchestra.run.smithi003.stdout:Need to get 207 MB of archives. 2023-03-03T06:20:16.236 INFO:teuthology.orchestra.run.smithi003.stdout:After this operation, 1,025 MB of additional disk space will be used. 2023-03-03T06:20:16.236 INFO:teuthology.orchestra.run.smithi003.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-03-03T06:20:16.242 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2023-03-03T06:20:16.242 INFO:teuthology.orchestra.run.smithi032.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-03-03T06:20:16.242 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-03-03T06:20:16.242 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-03-03T06:20:16.242 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-03-03T06:20:16.243 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-03-03T06:20:16.243 INFO:teuthology.orchestra.run.smithi032.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-03-03T06:20:16.243 INFO:teuthology.orchestra.run.smithi032.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-03-03T06:20:16.243 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-03-03T06:20:16.244 INFO:teuthology.orchestra.run.smithi032.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-03-03T06:20:16.244 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-03-03T06:20:16.244 INFO:teuthology.orchestra.run.smithi032.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-03-03T06:20:16.244 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2023-03-03T06:20:16.244 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-03-03T06:20:16.244 INFO:teuthology.orchestra.run.smithi032.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-03-03T06:20:16.244 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-03-03T06:20:16.245 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-03-03T06:20:16.245 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-03-03T06:20:16.245 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-03-03T06:20:16.245 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-03-03T06:20:16.245 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-03-03T06:20:16.245 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-03-03T06:20:16.245 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-03-03T06:20:16.246 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-03-03T06:20:16.246 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-03-03T06:20:16.246 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-03-03T06:20:16.246 INFO:teuthology.orchestra.run.smithi032.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-03-03T06:20:16.246 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2023-03-03T06:20:16.246 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 librbd1 2023-03-03T06:20:16.256 INFO:teuthology.orchestra.run.smithi003.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-03-03T06:20:16.264 INFO:teuthology.orchestra.run.smithi003.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-03-03T06:20:16.292 INFO:teuthology.orchestra.run.smithi003.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-03-03T06:20:16.320 INFO:teuthology.orchestra.run.smithi003.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-03-03T06:20:16.323 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-03-03T06:20:16.329 INFO:teuthology.orchestra.run.smithi003.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.0.0-2739-g472aaef4-1focal [3,430 kB] 2023-03-03T06:20:16.337 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-03-03T06:20:16.380 INFO:teuthology.orchestra.run.smithi003.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-03-03T06:20:16.383 INFO:teuthology.orchestra.run.smithi003.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-03-03T06:20:16.392 INFO:teuthology.orchestra.run.smithi003.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-03-03T06:20:16.394 INFO:teuthology.orchestra.run.smithi003.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-03-03T06:20:16.431 INFO:teuthology.orchestra.run.smithi003.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-03-03T06:20:16.435 INFO:teuthology.orchestra.run.smithi032.stdout:2 upgraded, 116 newly installed, 0 to remove and 152 not upgraded. 2023-03-03T06:20:16.436 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 207 MB of archives. 2023-03-03T06:20:16.436 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 1,025 MB of additional disk space will be used. 2023-03-03T06:20:16.436 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.0.0-2739-g472aaef4-1focal [3,430 kB] 2023-03-03T06:20:16.469 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-03-03T06:20:16.509 INFO:teuthology.orchestra.run.smithi003.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-03-03T06:20:16.513 INFO:teuthology.orchestra.run.smithi003.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-03-03T06:20:16.552 INFO:teuthology.orchestra.run.smithi003.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-03-03T06:20:16.553 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-03-03T06:20:16.554 INFO:teuthology.orchestra.run.smithi003.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-03-03T06:20:16.560 INFO:teuthology.orchestra.run.smithi003.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-03-03T06:20:16.561 INFO:teuthology.orchestra.run.smithi003.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-03-03T06:20:16.562 INFO:teuthology.orchestra.run.smithi003.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-03-03T06:20:16.563 INFO:teuthology.orchestra.run.smithi003.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-03-03T06:20:16.565 INFO:teuthology.orchestra.run.smithi003.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-03-03T06:20:16.565 INFO:teuthology.orchestra.run.smithi003.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-03-03T06:20:16.588 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-03-03T06:20:16.602 INFO:teuthology.orchestra.run.smithi003.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-03-03T06:20:16.608 INFO:teuthology.orchestra.run.smithi003.stdout:Get:25 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.0.0-2739-g472aaef4-1focal [3,983 kB] 2023-03-03T06:20:16.675 INFO:teuthology.orchestra.run.smithi003.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-03-03T06:20:16.679 INFO:teuthology.orchestra.run.smithi003.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-03-03T06:20:16.694 INFO:teuthology.orchestra.run.smithi003.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-03-03T06:20:16.706 INFO:teuthology.orchestra.run.smithi003.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-03-03T06:20:16.708 INFO:teuthology.orchestra.run.smithi003.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-03-03T06:20:16.709 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-03-03T06:20:16.735 INFO:teuthology.orchestra.run.smithi003.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-03-03T06:20:16.735 INFO:teuthology.orchestra.run.smithi003.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-03-03T06:20:16.736 INFO:teuthology.orchestra.run.smithi003.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-03-03T06:20:16.738 INFO:teuthology.orchestra.run.smithi003.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-03-03T06:20:16.739 INFO:teuthology.orchestra.run.smithi003.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2023-03-03T06:20:16.753 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.0.0-2739-g472aaef4-1focal [3,983 kB] 2023-03-03T06:20:16.780 INFO:teuthology.orchestra.run.smithi003.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-03-03T06:20:16.827 INFO:teuthology.orchestra.run.smithi003.stdout:Get:37 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.0.0-2739-g472aaef4-1focal [721 kB] 2023-03-03T06:20:16.847 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-03-03T06:20:16.863 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-03-03T06:20:16.867 INFO:teuthology.orchestra.run.smithi003.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.0.0-2739-g472aaef4-1focal [375 kB] 2023-03-03T06:20:16.874 INFO:teuthology.orchestra.run.smithi003.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-03-03T06:20:16.877 INFO:teuthology.orchestra.run.smithi003.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2023-03-03T06:20:16.879 INFO:teuthology.orchestra.run.smithi003.stdout:Get:41 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.0.0-2739-g472aaef4-1focal [22.4 kB] 2023-03-03T06:20:16.879 INFO:teuthology.orchestra.run.smithi003.stdout:Get:42 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.0.0-2739-g472aaef4-1focal [185 kB] 2023-03-03T06:20:16.886 INFO:teuthology.orchestra.run.smithi003.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-03-03T06:20:16.896 INFO:teuthology.orchestra.run.smithi003.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-03-03T06:20:16.898 INFO:teuthology.orchestra.run.smithi003.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-03-03T06:20:16.899 INFO:teuthology.orchestra.run.smithi003.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-03-03T06:20:16.901 INFO:teuthology.orchestra.run.smithi003.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-03-03T06:20:16.901 INFO:teuthology.orchestra.run.smithi003.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-03-03T06:20:16.902 INFO:teuthology.orchestra.run.smithi003.stdout:Get:49 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.0.0-2739-g472aaef4-1focal [47.7 kB] 2023-03-03T06:20:16.905 INFO:teuthology.orchestra.run.smithi003.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-03-03T06:20:16.905 INFO:teuthology.orchestra.run.smithi003.stdout:Get:51 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.0.0-2739-g472aaef4-1focal [354 kB] 2023-03-03T06:20:16.906 INFO:teuthology.orchestra.run.smithi003.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-03-03T06:20:16.915 INFO:teuthology.orchestra.run.smithi003.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.0.0-2739-g472aaef4-1focal [6,520 kB] 2023-03-03T06:20:16.925 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-03-03T06:20:16.942 INFO:teuthology.orchestra.run.smithi003.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-03-03T06:20:17.036 INFO:teuthology.orchestra.run.smithi003.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-03-03T06:20:17.037 INFO:teuthology.orchestra.run.smithi003.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-03-03T06:20:17.044 INFO:teuthology.orchestra.run.smithi003.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-03-03T06:20:17.054 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.0.0-2739-g472aaef4-1focal [721 kB] 2023-03-03T06:20:17.055 INFO:teuthology.orchestra.run.smithi003.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-03-03T06:20:17.061 INFO:teuthology.orchestra.run.smithi003.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-03-03T06:20:17.062 INFO:teuthology.orchestra.run.smithi003.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-03-03T06:20:17.063 INFO:teuthology.orchestra.run.smithi003.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-03-03T06:20:17.103 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.0.0-2739-g472aaef4-1focal [375 kB] 2023-03-03T06:20:17.115 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.0.0-2739-g472aaef4-1focal [22.4 kB] 2023-03-03T06:20:17.123 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-03-03T06:20:17.133 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-03-03T06:20:17.141 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.0.0-2739-g472aaef4-1focal [185 kB] 2023-03-03T06:20:17.149 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.0.0-2739-g472aaef4-1focal [47.7 kB] 2023-03-03T06:20:17.151 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.0.0-2739-g472aaef4-1focal [354 kB] 2023-03-03T06:20:17.158 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-03-03T06:20:17.163 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.0.0-2739-g472aaef4-1focal [6,520 kB] 2023-03-03T06:20:17.166 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-03-03T06:20:17.210 INFO:teuthology.orchestra.run.smithi003.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-03-03T06:20:17.235 INFO:teuthology.orchestra.run.smithi003.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-03-03T06:20:17.237 INFO:teuthology.orchestra.run.smithi003.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-03-03T06:20:17.291 INFO:teuthology.orchestra.run.smithi003.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-03-03T06:20:17.296 INFO:teuthology.orchestra.run.smithi003.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.0.0-2739-g472aaef4-1focal [103 kB] 2023-03-03T06:20:17.297 INFO:teuthology.orchestra.run.smithi003.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.0.0-2739-g472aaef4-1focal [467 kB] 2023-03-03T06:20:17.312 INFO:teuthology.orchestra.run.smithi003.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-03-03T06:20:17.328 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-03-03T06:20:17.338 INFO:teuthology.orchestra.run.smithi003.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.0.0-2739-g472aaef4-1focal [25.0 MB] 2023-03-03T06:20:17.343 INFO:teuthology.orchestra.run.smithi003.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-03-03T06:20:17.354 INFO:teuthology.orchestra.run.smithi003.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-03-03T06:20:17.360 INFO:teuthology.orchestra.run.smithi003.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-03-03T06:20:17.368 INFO:teuthology.orchestra.run.smithi003.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-03-03T06:20:17.370 INFO:teuthology.orchestra.run.smithi003.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-03-03T06:20:17.382 INFO:teuthology.orchestra.run.smithi003.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-03-03T06:20:17.383 INFO:teuthology.orchestra.run.smithi003.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-03-03T06:20:17.386 INFO:teuthology.orchestra.run.smithi003.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-03-03T06:20:17.404 INFO:teuthology.orchestra.run.smithi003.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-03-03T06:20:17.446 INFO:teuthology.orchestra.run.smithi003.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-03-03T06:20:17.527 INFO:teuthology.orchestra.run.smithi003.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-03-03T06:20:17.532 INFO:teuthology.orchestra.run.smithi003.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-03-03T06:20:17.565 INFO:teuthology.orchestra.run.smithi003.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-03-03T06:20:17.577 INFO:teuthology.orchestra.run.smithi003.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-03-03T06:20:17.578 INFO:teuthology.orchestra.run.smithi003.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-03-03T06:20:17.616 INFO:teuthology.orchestra.run.smithi003.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-03-03T06:20:17.620 INFO:teuthology.orchestra.run.smithi003.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2023-03-03T06:20:17.625 INFO:teuthology.orchestra.run.smithi003.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-03-03T06:20:17.659 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.0.0-2739-g472aaef4-1focal [103 kB] 2023-03-03T06:20:17.661 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.0.0-2739-g472aaef4-1focal [467 kB] 2023-03-03T06:20:17.681 INFO:teuthology.orchestra.run.smithi003.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-03-03T06:20:17.682 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-03-03T06:20:17.682 INFO:teuthology.orchestra.run.smithi003.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-03-03T06:20:17.699 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-03-03T06:20:17.716 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.0.0-2739-g472aaef4-1focal [25.0 MB] 2023-03-03T06:20:17.718 INFO:teuthology.orchestra.run.smithi003.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-03-03T06:20:17.793 INFO:teuthology.orchestra.run.smithi003.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-03-03T06:20:17.870 INFO:teuthology.orchestra.run.smithi003.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-03-03T06:20:17.871 INFO:teuthology.orchestra.run.smithi003.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-03-03T06:20:17.873 INFO:teuthology.orchestra.run.smithi003.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-03-03T06:20:17.873 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-03-03T06:20:17.874 INFO:teuthology.orchestra.run.smithi003.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-03-03T06:20:17.874 INFO:teuthology.orchestra.run.smithi003.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-03-03T06:20:17.883 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-03-03T06:20:17.915 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-03-03T06:20:17.921 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-03-03T06:20:17.924 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-03-03T06:20:17.926 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-03-03T06:20:17.935 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-03-03T06:20:17.936 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-03-03T06:20:18.099 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-03-03T06:20:18.412 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-03-03T06:20:18.435 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-03-03T06:20:18.505 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-03-03T06:20:18.568 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-03-03T06:20:18.575 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-03-03T06:20:18.693 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-03-03T06:20:18.694 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-03-03T06:20:18.698 INFO:teuthology.orchestra.run.smithi032.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-03-03T06:20:18.701 INFO:teuthology.orchestra.run.smithi032.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-03-03T06:20:18.712 INFO:teuthology.orchestra.run.smithi032.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2023-03-03T06:20:18.782 INFO:teuthology.orchestra.run.smithi003.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.0.0-2739-g472aaef4-1focal [11.0 MB] 2023-03-03T06:20:18.894 INFO:teuthology.orchestra.run.smithi032.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-03-03T06:20:19.320 INFO:teuthology.orchestra.run.smithi032.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-03-03T06:20:19.332 INFO:teuthology.orchestra.run.smithi032.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2023-03-03T06:20:19.374 INFO:teuthology.orchestra.run.smithi032.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-03-03T06:20:19.395 INFO:teuthology.orchestra.run.smithi003.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.0.0-2739-g472aaef4-1focal [221 kB] 2023-03-03T06:20:19.418 INFO:teuthology.orchestra.run.smithi032.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-03-03T06:20:19.425 INFO:teuthology.orchestra.run.smithi003.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.0.0-2739-g472aaef4-1focal [133 kB] 2023-03-03T06:20:19.426 INFO:teuthology.orchestra.run.smithi032.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-03-03T06:20:19.430 INFO:teuthology.orchestra.run.smithi003.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.0.0-2739-g472aaef4-1focal [1,546 kB] 2023-03-03T06:20:19.431 INFO:teuthology.orchestra.run.smithi032.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-03-03T06:20:19.441 INFO:teuthology.orchestra.run.smithi032.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-03-03T06:20:19.446 INFO:teuthology.orchestra.run.smithi032.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-03-03T06:20:19.462 INFO:teuthology.orchestra.run.smithi032.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-03-03T06:20:19.463 INFO:teuthology.orchestra.run.smithi032.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-03-03T06:20:19.511 INFO:teuthology.orchestra.run.smithi003.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.0.0-2739-g472aaef4-1focal [9,378 kB] 2023-03-03T06:20:19.631 INFO:teuthology.orchestra.run.smithi032.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-03-03T06:20:19.632 INFO:teuthology.orchestra.run.smithi032.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.0.0-2739-g472aaef4-1focal [11.0 MB] 2023-03-03T06:20:20.036 INFO:teuthology.orchestra.run.smithi032.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-03-03T06:20:20.039 INFO:teuthology.orchestra.run.smithi032.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-03-03T06:20:20.074 INFO:teuthology.orchestra.run.smithi032.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-03-03T06:20:20.077 INFO:teuthology.orchestra.run.smithi003.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.0.0-2739-g472aaef4-1focal [31.3 MB] 2023-03-03T06:20:20.121 INFO:teuthology.orchestra.run.smithi032.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-03-03T06:20:20.151 INFO:teuthology.orchestra.run.smithi032.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-03-03T06:20:20.155 INFO:teuthology.orchestra.run.smithi032.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-03-03T06:20:20.164 INFO:teuthology.orchestra.run.smithi032.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-03-03T06:20:20.433 INFO:teuthology.orchestra.run.smithi032.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.0.0-2739-g472aaef4-1focal [221 kB] 2023-03-03T06:20:20.475 INFO:teuthology.orchestra.run.smithi032.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.0.0-2739-g472aaef4-1focal [133 kB] 2023-03-03T06:20:20.478 INFO:teuthology.orchestra.run.smithi032.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.0.0-2739-g472aaef4-1focal [1,546 kB] 2023-03-03T06:20:20.574 INFO:teuthology.orchestra.run.smithi032.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.0.0-2739-g472aaef4-1focal [9,378 kB] 2023-03-03T06:20:20.650 INFO:teuthology.orchestra.run.smithi032.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-03-03T06:20:20.699 INFO:teuthology.orchestra.run.smithi032.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-03-03T06:20:20.701 INFO:teuthology.orchestra.run.smithi032.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-03-03T06:20:20.931 INFO:teuthology.orchestra.run.smithi032.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-03-03T06:20:21.014 INFO:teuthology.orchestra.run.smithi032.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-03-03T06:20:21.134 INFO:teuthology.orchestra.run.smithi032.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-03-03T06:20:21.190 INFO:teuthology.orchestra.run.smithi032.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-03-03T06:20:21.217 INFO:teuthology.orchestra.run.smithi032.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-03-03T06:20:21.250 INFO:teuthology.orchestra.run.smithi032.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-03-03T06:20:21.263 INFO:teuthology.orchestra.run.smithi032.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-03-03T06:20:21.313 INFO:teuthology.orchestra.run.smithi032.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-03-03T06:20:21.321 INFO:teuthology.orchestra.run.smithi032.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-03-03T06:20:21.322 INFO:teuthology.orchestra.run.smithi032.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.0.0-2739-g472aaef4-1focal [31.3 MB] 2023-03-03T06:20:21.334 INFO:teuthology.orchestra.run.smithi032.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-03-03T06:20:21.411 INFO:teuthology.orchestra.run.smithi032.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-03-03T06:20:21.597 INFO:teuthology.orchestra.run.smithi032.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-03-03T06:20:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.0.0-2739-g472aaef4-1focal [3,816 B] 2023-03-03T06:20:21.867 INFO:teuthology.orchestra.run.smithi003.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.0.0-2739-g472aaef4-1focal [829 kB] 2023-03-03T06:20:21.909 INFO:teuthology.orchestra.run.smithi003.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.0.0-2739-g472aaef4-1focal [2,286 kB] 2023-03-03T06:20:21.953 INFO:teuthology.orchestra.run.smithi032.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-03-03T06:20:21.977 INFO:teuthology.orchestra.run.smithi032.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-03-03T06:20:22.024 INFO:teuthology.orchestra.run.smithi003.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.0.0-2739-g472aaef4-1focal [96.8 kB] 2023-03-03T06:20:22.038 INFO:teuthology.orchestra.run.smithi003.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.0.0-2739-g472aaef4-1focal [117 kB] 2023-03-03T06:20:22.043 INFO:teuthology.orchestra.run.smithi003.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.0.0-2739-g472aaef4-1focal [1,795 kB] 2023-03-03T06:20:22.125 INFO:teuthology.orchestra.run.smithi032.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-03-03T06:20:22.128 INFO:teuthology.orchestra.run.smithi003.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.0.0-2739-g472aaef4-1focal [7,888 kB] 2023-03-03T06:20:22.178 INFO:teuthology.orchestra.run.smithi032.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-03-03T06:20:22.183 INFO:teuthology.orchestra.run.smithi032.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-03-03T06:20:22.346 INFO:teuthology.orchestra.run.smithi032.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-03-03T06:20:22.354 INFO:teuthology.orchestra.run.smithi032.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2023-03-03T06:20:22.363 INFO:teuthology.orchestra.run.smithi032.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-03-03T06:20:22.534 INFO:teuthology.orchestra.run.smithi032.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-03-03T06:20:22.534 INFO:teuthology.orchestra.run.smithi032.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-03-03T06:20:22.540 INFO:teuthology.orchestra.run.smithi003.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.0.0-2739-g472aaef4-1focal [3,936 B] 2023-03-03T06:20:22.541 INFO:teuthology.orchestra.run.smithi003.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.0.0-2739-g472aaef4-1focal [49.8 MB] 2023-03-03T06:20:22.696 INFO:teuthology.orchestra.run.smithi032.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-03-03T06:20:23.006 INFO:teuthology.orchestra.run.smithi032.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-03-03T06:20:23.365 INFO:teuthology.orchestra.run.smithi032.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-03-03T06:20:23.368 INFO:teuthology.orchestra.run.smithi032.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-03-03T06:20:23.379 INFO:teuthology.orchestra.run.smithi032.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-03-03T06:20:23.380 INFO:teuthology.orchestra.run.smithi032.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-03-03T06:20:23.384 INFO:teuthology.orchestra.run.smithi032.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-03-03T06:20:24.067 INFO:teuthology.orchestra.run.smithi032.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.0.0-2739-g472aaef4-1focal [3,816 B] 2023-03-03T06:20:24.068 INFO:teuthology.orchestra.run.smithi032.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.0.0-2739-g472aaef4-1focal [829 kB] 2023-03-03T06:20:24.121 INFO:teuthology.orchestra.run.smithi032.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.0.0-2739-g472aaef4-1focal [2,286 kB] 2023-03-03T06:20:24.328 INFO:teuthology.orchestra.run.smithi032.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.0.0-2739-g472aaef4-1focal [96.8 kB] 2023-03-03T06:20:24.331 INFO:teuthology.orchestra.run.smithi032.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.0.0-2739-g472aaef4-1focal [117 kB] 2023-03-03T06:20:24.335 INFO:teuthology.orchestra.run.smithi032.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.0.0-2739-g472aaef4-1focal [1,795 kB] 2023-03-03T06:20:24.505 INFO:teuthology.orchestra.run.smithi032.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.0.0-2739-g472aaef4-1focal [7,888 kB] 2023-03-03T06:20:25.118 INFO:teuthology.orchestra.run.smithi003.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.0.0-2739-g472aaef4-1focal [225 kB] 2023-03-03T06:20:25.121 INFO:teuthology.orchestra.run.smithi003.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.0.0-2739-g472aaef4-1focal [253 kB] 2023-03-03T06:20:25.128 INFO:teuthology.orchestra.run.smithi003.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.0.0-2739-g472aaef4-1focal [23.3 kB] 2023-03-03T06:20:25.129 INFO:teuthology.orchestra.run.smithi003.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.0.0-2739-g472aaef4-1focal [13.0 kB] 2023-03-03T06:20:25.129 INFO:teuthology.orchestra.run.smithi003.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.0.0-2739-g472aaef4-1focal [27.4 kB] 2023-03-03T06:20:25.130 INFO:teuthology.orchestra.run.smithi003.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.0.0-2739-g472aaef4-1focal [13.3 MB] 2023-03-03T06:20:25.191 INFO:teuthology.orchestra.run.smithi032.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.0.0-2739-g472aaef4-1focal [3,936 B] 2023-03-03T06:20:25.191 INFO:teuthology.orchestra.run.smithi032.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.0.0-2739-g472aaef4-1focal [49.8 MB] 2023-03-03T06:20:25.836 INFO:teuthology.orchestra.run.smithi003.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.0.0-2739-g472aaef4-1focal [97.2 kB] 2023-03-03T06:20:26.944 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 207 MB in 10s (21.4 MB/s) 2023-03-03T06:20:27.083 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyperclip. 2023-03-03T06:20:27.162 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107774 files and directories currently installed.) 2023-03-03T06:20:27.164 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-03-03T06:20:27.198 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-03-03T06:20:27.249 INFO:teuthology.orchestra.run.smithi032.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.0.0-2739-g472aaef4-1focal [225 kB] 2023-03-03T06:20:27.252 INFO:teuthology.orchestra.run.smithi032.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.0.0-2739-g472aaef4-1focal [253 kB] 2023-03-03T06:20:27.255 INFO:teuthology.orchestra.run.smithi032.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.0.0-2739-g472aaef4-1focal [23.3 kB] 2023-03-03T06:20:27.255 INFO:teuthology.orchestra.run.smithi032.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.0.0-2739-g472aaef4-1focal [13.0 kB] 2023-03-03T06:20:27.256 INFO:teuthology.orchestra.run.smithi032.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.0.0-2739-g472aaef4-1focal [27.4 kB] 2023-03-03T06:20:27.256 INFO:teuthology.orchestra.run.smithi032.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.0.0-2739-g472aaef4-1focal [13.3 MB] 2023-03-03T06:20:27.543 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-wcwidth. 2023-03-03T06:20:27.563 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-03-03T06:20:27.600 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-03-03T06:20:27.694 INFO:teuthology.orchestra.run.smithi032.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230303.023823/472aaef4ec3b3a2c29f73ca67854785a910f1f9b/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.0.0-2739-g472aaef4-1focal [97.2 kB] 2023-03-03T06:20:27.995 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cmd2. 2023-03-03T06:20:28.016 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-03-03T06:20:28.053 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-03-03T06:20:28.440 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-03-03T06:20:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-03-03T06:20:28.540 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-03-03T06:20:28.834 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 207 MB in 11s (18.1 MB/s) 2023-03-03T06:20:28.985 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-03-03T06:20:29.005 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-03-03T06:20:29.014 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyperclip. 2023-03-03T06:20:29.073 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-03-03T06:20:29.094 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107774 files and directories currently installed.) 2023-03-03T06:20:29.097 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-03-03T06:20:29.137 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-03-03T06:20:29.638 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-03-03T06:20:29.659 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-03-03T06:20:29.696 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-03-03T06:20:29.783 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-wcwidth. 2023-03-03T06:20:29.796 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-03-03T06:20:29.840 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-03-03T06:20:30.158 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-03-03T06:20:30.179 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-03-03T06:20:30.243 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-03-03T06:20:30.244 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cmd2. 2023-03-03T06:20:30.264 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-03-03T06:20:30.302 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-03-03T06:20:30.672 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-03-03T06:20:30.691 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-03-03T06:20:30.772 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-03-03T06:20:30.803 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-03-03T06:20:30.822 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-03-03T06:20:30.861 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-03-03T06:20:31.167 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-03-03T06:20:31.187 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-03-03T06:20:31.225 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-03-03T06:20:31.289 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-03-03T06:20:31.310 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-03-03T06:20:31.347 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-03-03T06:20:31.645 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-03-03T06:20:31.666 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-03-03T06:20:31.704 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-03-03T06:20:31.800 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-03-03T06:20:31.821 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-03-03T06:20:31.858 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-03-03T06:20:32.207 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-03-03T06:20:32.226 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-03-03T06:20:32.282 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../010-librbd1_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:32.303 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-03-03T06:20:32.403 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librbd1 (18.0.0-2739-g472aaef4-1focal) over (15.2.17-0ubuntu0.20.04.1) ... 2023-03-03T06:20:33.052 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-03-03T06:20:33.073 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-03-03T06:20:33.119 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-03-03T06:20:33.237 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../011-librados2_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:33.509 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librados2 (18.0.0-2739-g472aaef4-1focal) over (15.2.17-0ubuntu0.20.04.1) ... 2023-03-03T06:20:33.539 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-03-03T06:20:33.558 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-03-03T06:20:33.597 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-03-03T06:20:34.025 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-03-03T06:20:34.045 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-03-03T06:20:34.083 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-03-03T06:20:34.178 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libcephfs2. 2023-03-03T06:20:34.192 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:34.237 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libcephfs2 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:34.507 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../010-librbd1_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:34.606 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rados. 2023-03-03T06:20:34.627 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../013-python3-rados_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:34.664 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rados (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:34.895 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librbd1 (18.0.0-2739-g472aaef4-1focal) over (15.2.17-0ubuntu0.20.04.1) ... 2023-03-03T06:20:35.042 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-03-03T06:20:35.063 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:20:35.100 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-ceph-argparse (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:35.411 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cephfs. 2023-03-03T06:20:35.431 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:35.469 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cephfs (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:35.695 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../011-librados2_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:35.816 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librados2 (18.0.0-2739-g472aaef4-1focal) over (15.2.17-0ubuntu0.20.04.1) ... 2023-03-03T06:20:35.864 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-ceph-common. 2023-03-03T06:20:35.884 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:20:35.921 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-ceph-common (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:36.300 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-prettytable. 2023-03-03T06:20:36.320 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2023-03-03T06:20:36.358 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-03-03T06:20:36.620 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs2. 2023-03-03T06:20:36.641 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:36.660 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rbd. 2023-03-03T06:20:36.678 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs2 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:36.682 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:36.728 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rbd (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:37.106 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rados. 2023-03-03T06:20:37.126 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../013-python3-rados_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:37.146 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-03-03T06:20:37.164 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rados (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:37.167 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-03-03T06:20:37.204 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-03-03T06:20:37.551 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-03-03T06:20:37.570 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:20:37.617 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-argparse (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:37.633 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-03-03T06:20:37.653 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2023-03-03T06:20:37.691 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-03-03T06:20:37.928 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cephfs. 2023-03-03T06:20:37.949 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:37.987 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cephfs (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:38.075 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-03-03T06:20:38.096 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-03-03T06:20:38.141 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-03-03T06:20:38.357 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-common. 2023-03-03T06:20:38.377 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:20:38.414 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-common (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:38.536 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package librgw2. 2023-03-03T06:20:38.557 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../022-librgw2_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:38.594 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librgw2 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:38.810 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-prettytable. 2023-03-03T06:20:38.830 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2023-03-03T06:20:38.867 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-03-03T06:20:39.212 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rbd. 2023-03-03T06:20:39.233 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:39.270 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rbd (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:39.555 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rgw. 2023-03-03T06:20:39.575 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../023-python3-rgw_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:39.613 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rgw (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:39.665 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-03-03T06:20:39.686 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-03-03T06:20:39.731 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-03-03T06:20:40.049 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liboath0. 2023-03-03T06:20:40.070 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-03-03T06:20:40.107 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-03-03T06:20:40.143 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-03-03T06:20:40.163 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2023-03-03T06:20:40.201 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-03-03T06:20:40.503 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libradosstriper1. 2023-03-03T06:20:40.526 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../025-libradosstriper1_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:40.561 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libradosstriper1 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:40.602 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-03-03T06:20:40.622 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-03-03T06:20:40.659 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-03-03T06:20:41.106 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librgw2. 2023-03-03T06:20:41.115 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-common. 2023-03-03T06:20:41.126 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../022-librgw2_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:41.135 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../026-ceph-common_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:41.163 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librgw2 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:41.173 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-common (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:42.091 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rgw. 2023-03-03T06:20:42.110 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../023-python3-rgw_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:42.149 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rgw (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:42.535 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liboath0. 2023-03-03T06:20:42.556 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-03-03T06:20:42.602 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-03-03T06:20:43.006 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libradosstriper1. 2023-03-03T06:20:43.027 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../025-libradosstriper1_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:43.073 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libradosstriper1 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:43.593 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-common. 2023-03-03T06:20:43.614 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../026-ceph-common_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:43.659 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-common (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:43.881 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-base. 2023-03-03T06:20:43.900 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../027-ceph-base_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:43.978 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-base (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:45.536 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cherrypy3. 2023-03-03T06:20:45.557 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-03-03T06:20:45.602 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-03-03T06:20:46.048 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-natsort. 2023-03-03T06:20:46.068 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2023-03-03T06:20:46.097 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-03-03T06:20:46.367 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-base. 2023-03-03T06:20:46.381 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../027-ceph-base_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:46.461 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-base (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:46.484 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-logutils. 2023-03-03T06:20:46.505 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-03-03T06:20:46.542 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-03-03T06:20:46.895 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-markupsafe. 2023-03-03T06:20:46.916 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-03-03T06:20:46.953 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-03-03T06:20:47.306 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-mako. 2023-03-03T06:20:47.320 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-03-03T06:20:47.364 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-03-03T06:20:47.793 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-simplegeneric. 2023-03-03T06:20:47.812 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-03-03T06:20:47.850 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-03-03T06:20:48.033 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cherrypy3. 2023-03-03T06:20:48.054 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-03-03T06:20:48.100 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-03-03T06:20:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-singledispatch. 2023-03-03T06:20:48.216 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-03-03T06:20:48.253 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-03-03T06:20:48.512 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-natsort. 2023-03-03T06:20:48.532 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2023-03-03T06:20:48.578 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-03-03T06:20:48.648 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-webob. 2023-03-03T06:20:48.669 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-03-03T06:20:48.706 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-03-03T06:20:48.973 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-logutils. 2023-03-03T06:20:48.993 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-03-03T06:20:49.031 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-03-03T06:20:49.101 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-soupsieve. 2023-03-03T06:20:49.122 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-03-03T06:20:49.159 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-03-03T06:20:49.426 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-markupsafe. 2023-03-03T06:20:49.446 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-03-03T06:20:49.484 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-03-03T06:20:49.503 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-bs4. 2023-03-03T06:20:49.523 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-03-03T06:20:49.878 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-mako. 2023-03-03T06:20:49.899 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-03-03T06:20:50.103 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-03-03T06:20:50.120 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-03-03T06:20:50.523 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-waitress. 2023-03-03T06:20:50.544 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-03-03T06:20:50.596 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-03-03T06:20:50.899 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplegeneric. 2023-03-03T06:20:50.919 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-03-03T06:20:50.956 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-03-03T06:20:51.018 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-tempita. 2023-03-03T06:20:51.039 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-03-03T06:20:51.085 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-03-03T06:20:51.310 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-singledispatch. 2023-03-03T06:20:51.330 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-03-03T06:20:51.367 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-03-03T06:20:51.438 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-paste. 2023-03-03T06:20:51.459 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-03-03T06:20:51.505 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-03-03T06:20:51.720 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webob. 2023-03-03T06:20:51.742 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-03-03T06:20:51.786 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-03-03T06:20:51.901 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-03-03T06:20:51.921 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-03-03T06:20:51.959 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-03-03T06:20:52.198 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-soupsieve. 2023-03-03T06:20:52.218 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-03-03T06:20:52.256 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-03-03T06:20:52.309 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pastedeploy. 2023-03-03T06:20:52.329 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-03-03T06:20:52.392 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-03-03T06:20:52.634 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-bs4. 2023-03-03T06:20:52.654 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-03-03T06:20:52.692 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-03-03T06:20:52.762 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-webtest. 2023-03-03T06:20:52.783 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-03-03T06:20:52.828 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-03-03T06:20:53.120 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-waitress. 2023-03-03T06:20:53.141 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-03-03T06:20:53.204 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-03-03T06:20:53.224 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pecan. 2023-03-03T06:20:53.244 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-03-03T06:20:53.290 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-03-03T06:20:53.616 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempita. 2023-03-03T06:20:53.630 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-03-03T06:20:53.674 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-03-03T06:20:53.754 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-werkzeug. 2023-03-03T06:20:53.774 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2023-03-03T06:20:53.836 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2023-03-03T06:20:54.035 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-paste. 2023-03-03T06:20:54.049 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-03-03T06:20:54.094 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-03-03T06:20:54.189 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-03-03T06:20:54.205 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:20:54.247 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-modules-core (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:54.490 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-03-03T06:20:54.510 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-03-03T06:20:54.548 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-03-03T06:20:54.720 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-03-03T06:20:54.740 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:54.786 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:54.923 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastedeploy. 2023-03-03T06:20:54.942 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-03-03T06:20:54.981 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-03-03T06:20:55.156 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cryptography. 2023-03-03T06:20:55.176 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../048-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-03-03T06:20:55.222 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-03-03T06:20:55.334 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webtest. 2023-03-03T06:20:55.348 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-03-03T06:20:55.392 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-03-03T06:20:55.659 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-openssl. 2023-03-03T06:20:55.680 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../049-python3-openssl_19.0.0-1build1_all.deb ... 2023-03-03T06:20:55.717 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-03-03T06:20:55.754 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pecan. 2023-03-03T06:20:55.774 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-03-03T06:20:55.812 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-03-03T06:20:56.096 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr. 2023-03-03T06:20:56.116 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../050-ceph-mgr_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:56.162 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:56.275 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-werkzeug. 2023-03-03T06:20:56.295 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2023-03-03T06:20:56.333 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2023-03-03T06:20:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mon. 2023-03-03T06:20:56.719 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../051-ceph-mon_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:56.720 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-03-03T06:20:56.738 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:20:56.766 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mon (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:56.777 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-modules-core (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:57.258 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-03-03T06:20:57.277 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:57.316 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:57.719 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cryptography. 2023-03-03T06:20:57.733 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../048-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-03-03T06:20:57.776 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-03-03T06:20:58.086 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-osd. 2023-03-03T06:20:58.104 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../052-ceph-osd_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:58.144 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-osd (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:58.247 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-openssl. 2023-03-03T06:20:58.267 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../049-python3-openssl_19.0.0-1build1_all.deb ... 2023-03-03T06:20:58.314 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-03-03T06:20:58.759 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr. 2023-03-03T06:20:58.780 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../050-ceph-mgr_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:58.825 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:20:59.395 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mon. 2023-03-03T06:20:59.416 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../051-ceph-mon_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:20:59.461 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mon (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:00.865 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-osd. 2023-03-03T06:21:00.880 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../052-ceph-osd_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:00.922 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-osd (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:01.040 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph. 2023-03-03T06:21:01.060 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../053-ceph_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:01.098 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:01.476 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-fuse. 2023-03-03T06:21:01.497 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../054-ceph-fuse_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:01.542 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-fuse (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:01.979 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mds. 2023-03-03T06:21:02.001 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../055-ceph-mds_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:02.046 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mds (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:02.665 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package cephadm. 2023-03-03T06:21:02.685 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../056-cephadm_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:02.731 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking cephadm (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:03.109 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-asyncssh. 2023-03-03T06:21:03.129 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../057-python3-asyncssh_1.12.2-1_all.deb ... 2023-03-03T06:21:03.167 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2023-03-03T06:21:03.603 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-jinja2. 2023-03-03T06:21:03.624 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../058-python3-jinja2_2.10.1-2_all.deb ... 2023-03-03T06:21:03.669 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-03-03T06:21:03.786 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph. 2023-03-03T06:21:03.807 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../053-ceph_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:03.852 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:04.081 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-03-03T06:21:04.101 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../059-ceph-mgr-cephadm_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:04.246 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-fuse. 2023-03-03T06:21:04.554 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-cephadm (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:04.554 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../054-ceph-fuse_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:04.554 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-fuse (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:04.675 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-bcrypt. 2023-03-03T06:21:04.696 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../060-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-03-03T06:21:04.733 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-03-03T06:21:04.791 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mds. 2023-03-03T06:21:04.810 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../055-ceph-mds_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:04.849 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mds (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:05.079 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-jwt. 2023-03-03T06:21:05.099 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-03-03T06:21:05.144 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-03-03T06:21:05.469 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephadm. 2023-03-03T06:21:05.490 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../056-cephadm_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:05.531 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-attr. 2023-03-03T06:21:05.535 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephadm (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:05.552 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2023-03-03T06:21:05.597 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-03-03T06:21:06.113 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-asyncssh. 2023-03-03T06:21:06.125 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pluggy. 2023-03-03T06:21:06.133 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../057-python3-asyncssh_1.12.2-1_all.deb ... 2023-03-03T06:21:06.144 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2023-03-03T06:21:06.170 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2023-03-03T06:21:06.200 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-03-03T06:21:06.573 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-py. 2023-03-03T06:21:06.589 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-03-03T06:21:06.607 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jinja2. 2023-03-03T06:21:06.627 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-03-03T06:21:06.628 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../058-python3-jinja2_2.10.1-2_all.deb ... 2023-03-03T06:21:06.673 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-03-03T06:21:07.039 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-atomicwrites. 2023-03-03T06:21:07.059 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-03-03T06:21:07.068 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-03-03T06:21:07.089 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../059-ceph-mgr-cephadm_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:07.096 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-03-03T06:21:07.134 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-cephadm (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:07.563 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-bcrypt. 2023-03-03T06:21:07.584 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../060-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-03-03T06:21:07.599 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-packaging. 2023-03-03T06:21:07.620 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2023-03-03T06:21:07.629 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-03-03T06:21:07.665 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-packaging (20.3-1) ... 2023-03-03T06:21:07.999 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jwt. 2023-03-03T06:21:08.020 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-03-03T06:21:08.057 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-03-03T06:21:08.069 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pytest. 2023-03-03T06:21:08.089 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2023-03-03T06:21:08.126 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-03-03T06:21:08.452 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-attr. 2023-03-03T06:21:08.473 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2023-03-03T06:21:08.518 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-03-03T06:21:08.555 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-repoze.lru. 2023-03-03T06:21:08.577 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2023-03-03T06:21:08.621 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-03-03T06:21:08.913 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pluggy. 2023-03-03T06:21:08.932 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libjs-underscore. 2023-03-03T06:21:08.933 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2023-03-03T06:21:08.953 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-03-03T06:21:08.971 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-03-03T06:21:08.990 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-03-03T06:21:09.349 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-py. 2023-03-03T06:21:09.351 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-03-03T06:21:09.370 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-03-03T06:21:09.371 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-03-03T06:21:09.409 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-03-03T06:21:09.415 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-03-03T06:21:09.835 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-atomicwrites. 2023-03-03T06:21:09.856 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-03-03T06:21:09.923 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-03-03T06:21:09.937 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-routes. 2023-03-03T06:21:09.958 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2023-03-03T06:21:10.003 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-03-03T06:21:10.296 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-packaging. 2023-03-03T06:21:10.315 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2023-03-03T06:21:10.354 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-packaging (20.3-1) ... 2023-03-03T06:21:10.390 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-03-03T06:21:10.410 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:10.455 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-dashboard (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:10.757 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pytest. 2023-03-03T06:21:10.778 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2023-03-03T06:21:10.823 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-03-03T06:21:11.026 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-decorator. 2023-03-03T06:21:11.047 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-03-03T06:21:11.092 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-03-03T06:21:11.252 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-repoze.lru. 2023-03-03T06:21:11.272 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2023-03-03T06:21:11.318 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-03-03T06:21:11.445 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-03-03T06:21:11.465 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-03-03T06:21:11.503 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-03-03T06:21:11.646 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjs-underscore. 2023-03-03T06:21:11.667 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-03-03T06:21:11.720 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-03-03T06:21:11.872 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-scipy. 2023-03-03T06:21:11.893 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-03-03T06:21:11.930 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-03-03T06:21:12.082 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-03-03T06:21:12.102 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-03-03T06:21:12.148 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-03-03T06:21:12.526 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-routes. 2023-03-03T06:21:12.546 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2023-03-03T06:21:12.592 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-03-03T06:21:12.987 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-03-03T06:21:13.007 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:13.044 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-dashboard (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:13.472 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-03-03T06:21:13.493 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-03-03T06:21:13.538 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-03-03T06:21:13.640 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-decorator. 2023-03-03T06:21:13.659 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-03-03T06:21:13.698 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-03-03T06:21:14.076 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-03-03T06:21:14.094 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-03-03T06:21:14.134 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-03-03T06:21:14.167 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-joblib. 2023-03-03T06:21:14.188 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2023-03-03T06:21:14.233 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-03-03T06:21:14.487 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-scipy. 2023-03-03T06:21:14.506 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-03-03T06:21:14.536 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-03-03T06:21:14.695 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-sklearn. 2023-03-03T06:21:14.716 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-03-03T06:21:14.761 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-03-03T06:21:15.769 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-03-03T06:21:15.791 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:15.835 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:16.053 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-03-03T06:21:16.075 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-03-03T06:21:16.119 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-03-03T06:21:16.732 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-joblib. 2023-03-03T06:21:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2023-03-03T06:21:16.797 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-03-03T06:21:17.138 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cachetools. 2023-03-03T06:21:17.160 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2023-03-03T06:21:17.204 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-03-03T06:21:17.242 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn. 2023-03-03T06:21:17.257 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-03-03T06:21:17.300 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-03-03T06:21:17.802 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyasn1. 2023-03-03T06:21:17.820 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-03-03T06:21:17.859 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-03-03T06:21:18.159 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-03-03T06:21:18.179 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:18.217 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:18.255 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-03-03T06:21:18.271 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-03-03T06:21:18.313 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-03-03T06:21:18.700 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rsa. 2023-03-03T06:21:18.720 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2023-03-03T06:21:18.766 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rsa (4.0-3) ... 2023-03-03T06:21:19.278 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-google-auth. 2023-03-03T06:21:19.299 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2023-03-03T06:21:19.344 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-03-03T06:21:19.520 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cachetools. 2023-03-03T06:21:19.539 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2023-03-03T06:21:19.578 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-03-03T06:21:19.731 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-blinker. 2023-03-03T06:21:19.752 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-03-03T06:21:19.797 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-03-03T06:21:20.050 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1. 2023-03-03T06:21:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-03-03T06:21:20.116 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-03-03T06:21:20.167 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-oauthlib. 2023-03-03T06:21:20.187 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-03-03T06:21:20.225 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-03-03T06:21:20.537 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-03-03T06:21:20.559 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-03-03T06:21:20.603 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-03-03T06:21:20.654 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-03-03T06:21:20.675 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-03-03T06:21:20.719 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-03-03T06:21:20.999 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rsa. 2023-03-03T06:21:21.019 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2023-03-03T06:21:21.064 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rsa (4.0-3) ... 2023-03-03T06:21:21.114 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-websocket. 2023-03-03T06:21:21.136 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-03-03T06:21:21.180 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-03-03T06:21:21.460 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-google-auth. 2023-03-03T06:21:21.481 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2023-03-03T06:21:21.526 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-03-03T06:21:21.726 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-kubernetes. 2023-03-03T06:21:21.746 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-03-03T06:21:21.792 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-03-03T06:21:21.896 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-blinker. 2023-03-03T06:21:21.917 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-03-03T06:21:21.962 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-03-03T06:21:22.341 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-oauthlib. 2023-03-03T06:21:22.362 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-03-03T06:21:22.398 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-03-03T06:21:22.547 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-03-03T06:21:22.568 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:22.613 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:22.811 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-03-03T06:21:22.831 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-03-03T06:21:22.876 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-03-03T06:21:22.991 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libonig5:amd64. 2023-03-03T06:21:23.013 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2023-03-03T06:21:23.057 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-03-03T06:21:23.330 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-websocket. 2023-03-03T06:21:23.351 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-03-03T06:21:23.396 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-03-03T06:21:23.477 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libjq1:amd64. 2023-03-03T06:21:23.498 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-03-03T06:21:23.543 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-03-03T06:21:23.817 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-kubernetes. 2023-03-03T06:21:23.838 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-03-03T06:21:23.863 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package jq. 2023-03-03T06:21:23.882 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-03-03T06:21:23.884 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-03-03T06:21:23.929 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-03-03T06:21:24.357 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package socat. 2023-03-03T06:21:24.377 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2023-03-03T06:21:24.406 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking socat (1.7.3.3-2) ... 2023-03-03T06:21:24.646 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-03-03T06:21:24.667 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:24.712 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:24.777 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package xmlstarlet. 2023-03-03T06:21:24.797 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-03-03T06:21:24.842 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-03-03T06:21:25.115 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libonig5:amd64. 2023-03-03T06:21:25.137 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2023-03-03T06:21:25.181 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-03-03T06:21:25.289 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-test. 2023-03-03T06:21:25.310 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../096-ceph-test_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:25.355 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-test (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:25.585 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjq1:amd64. 2023-03-03T06:21:25.606 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-03-03T06:21:25.651 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-03-03T06:21:25.987 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package jq. 2023-03-03T06:21:26.008 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-03-03T06:21:26.045 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-03-03T06:21:26.373 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package socat. 2023-03-03T06:21:26.394 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2023-03-03T06:21:26.448 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking socat (1.7.3.3-2) ... 2023-03-03T06:21:26.810 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package xmlstarlet. 2023-03-03T06:21:26.830 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-03-03T06:21:26.876 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-03-03T06:21:27.339 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-test. 2023-03-03T06:21:27.359 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../096-ceph-test_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:27.404 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-test (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:29.809 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-volume. 2023-03-03T06:21:29.830 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../097-ceph-volume_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:29.875 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-volume (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:30.329 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package cephfs-mirror. 2023-03-03T06:21:30.351 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../098-cephfs-mirror_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:30.395 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking cephfs-mirror (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:30.798 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-colorama. 2023-03-03T06:21:30.820 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2023-03-03T06:21:30.864 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-03-03T06:21:31.209 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package cephfs-shell. 2023-03-03T06:21:31.230 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../100-cephfs-shell_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:31.275 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking cephfs-shell (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:31.603 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package cephfs-top. 2023-03-03T06:21:31.624 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../101-cephfs-top_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:31.669 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking cephfs-top (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:31.876 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-volume. 2023-03-03T06:21:31.896 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../097-ceph-volume_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:31.942 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-volume (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:31.972 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package fonts-mathjax. 2023-03-03T06:21:31.993 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-03-03T06:21:32.038 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-03-03T06:21:32.371 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-mirror. 2023-03-03T06:21:32.392 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../098-cephfs-mirror_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:32.437 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-mirror (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:32.634 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libcephfs-dev. 2023-03-03T06:21:32.655 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../103-libcephfs-dev_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:32.699 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libcephfs-dev (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:32.857 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-colorama. 2023-03-03T06:21:32.878 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2023-03-03T06:21:32.923 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-03-03T06:21:33.136 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package nvme-cli. 2023-03-03T06:21:33.158 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-03-03T06:21:33.261 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-03-03T06:21:33.285 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-shell. 2023-03-03T06:21:33.306 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../100-cephfs-shell_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:33.351 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-shell (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:33.670 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-top. 2023-03-03T06:21:33.682 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-03-03T06:21:33.692 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../101-cephfs-top_18.0.0-2739-g472aaef4-1focal_all.deb ... 2023-03-03T06:21:33.702 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2023-03-03T06:21:33.739 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2023-03-03T06:21:33.744 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-top (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:34.193 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libjs-mathjax. 2023-03-03T06:21:34.215 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-03-03T06:21:34.231 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package fonts-mathjax. 2023-03-03T06:21:34.251 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-03-03T06:21:34.259 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-03-03T06:21:34.288 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-03-03T06:21:34.877 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs-dev. 2023-03-03T06:21:35.522 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../103-libcephfs-dev_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:35.522 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs-dev (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:35.523 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package nvme-cli. 2023-03-03T06:21:35.523 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-03-03T06:21:35.523 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-03-03T06:21:35.950 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-03-03T06:21:35.972 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2023-03-03T06:21:36.016 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2023-03-03T06:21:36.719 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjs-mathjax. 2023-03-03T06:21:36.741 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-03-03T06:21:36.786 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-03-03T06:21:37.035 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-natsort-doc. 2023-03-03T06:21:37.059 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2023-03-03T06:21:37.101 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-03-03T06:21:37.479 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-webencodings. 2023-03-03T06:21:37.501 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-03-03T06:21:37.545 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-03-03T06:21:37.898 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-html5lib. 2023-03-03T06:21:37.913 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2023-03-03T06:21:37.956 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-03-03T06:21:38.343 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-03-03T06:21:38.366 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-03-03T06:21:38.409 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-03-03T06:21:38.980 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pastescript. 2023-03-03T06:21:39.001 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2023-03-03T06:21:39.046 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-03-03T06:21:39.203 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-natsort-doc. 2023-03-03T06:21:39.226 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2023-03-03T06:21:39.277 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-03-03T06:21:39.441 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-psutil. 2023-03-03T06:21:39.462 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-03-03T06:21:39.507 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-03-03T06:21:39.664 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webencodings. 2023-03-03T06:21:39.685 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-03-03T06:21:39.729 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-03-03T06:21:39.910 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyinotify. 2023-03-03T06:21:39.931 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-03-03T06:21:39.976 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-03-03T06:21:40.091 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-html5lib. 2023-03-03T06:21:40.113 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2023-03-03T06:21:40.157 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-03-03T06:21:40.346 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-simplejson. 2023-03-03T06:21:40.367 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-03-03T06:21:40.403 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-03-03T06:21:40.561 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-03-03T06:21:40.583 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-03-03T06:21:40.627 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-03-03T06:21:40.740 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-03-03T06:21:40.762 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-03-03T06:21:40.806 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-03-03T06:21:41.189 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastescript. 2023-03-03T06:21:41.212 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2023-03-03T06:21:41.255 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-03-03T06:21:41.552 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package radosgw. 2023-03-03T06:21:41.574 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../116-radosgw_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:41.617 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking radosgw (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:41.642 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-psutil. 2023-03-03T06:21:41.664 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-03-03T06:21:41.708 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-03-03T06:21:42.111 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyinotify. 2023-03-03T06:21:42.132 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-03-03T06:21:42.177 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-03-03T06:21:42.555 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplejson. 2023-03-03T06:21:42.576 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-03-03T06:21:42.621 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-03-03T06:21:43.007 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-03-03T06:21:43.030 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-03-03T06:21:43.062 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package rbd-fuse. 2023-03-03T06:21:43.073 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-03-03T06:21:43.077 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../117-rbd-fuse_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:43.120 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking rbd-fuse (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:43.501 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-attr (19.3.0-2) ... 2023-03-03T06:21:43.836 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package radosgw. 2023-03-03T06:21:43.858 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../116-radosgw_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:43.901 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking radosgw (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:43.917 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-03-03T06:21:44.043 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-03-03T06:21:44.169 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-03-03T06:21:44.411 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up cephadm (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:45.171 INFO:teuthology.orchestra.run.smithi003.stdout:Adding system user cephadm....done 2023-03-03T06:21:45.339 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package rbd-fuse. 2023-03-03T06:21:45.360 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../117-rbd-fuse_18.0.0-2739-g472aaef4-1focal_amd64.deb ... 2023-03-03T06:21:45.377 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-03-03T06:21:45.404 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking rbd-fuse (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:45.503 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-03-03T06:21:45.771 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-attr (19.3.0-2) ... 2023-03-03T06:21:45.997 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-03-03T06:21:46.192 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-03-03T06:21:46.318 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-03-03T06:21:46.423 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-03-03T06:21:46.453 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-03-03T06:21:46.549 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-03-03T06:21:46.675 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-03-03T06:21:46.704 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephadm (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:47.126 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-03-03T06:21:47.512 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user cephadm....done 2023-03-03T06:21:47.662 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-03-03T06:21:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-03-03T06:21:47.854 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-03-03T06:21:48.071 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-03-03T06:21:48.330 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-03-03T06:21:48.472 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-03-03T06:21:48.740 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-03-03T06:21:48.866 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-03-03T06:21:48.873 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-03-03T06:21:48.991 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-03-03T06:21:49.266 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-03-03T06:21:49.452 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-03-03T06:21:49.675 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-ceph-argparse (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:49.995 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-03-03T06:21:50.093 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-03-03T06:21:50.252 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-03-03T06:21:50.388 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-03-03T06:21:50.636 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-03-03T06:21:50.789 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-03-03T06:21:51.046 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-03-03T06:21:51.215 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-03-03T06:21:51.447 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-03-03T06:21:51.573 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-03-03T06:21:51.599 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-03-03T06:21:52.008 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-argparse (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:52.082 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-03-03T06:21:52.426 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-03-03T06:21:52.492 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-03-03T06:21:52.553 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-03-03T06:21:52.884 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-03-03T06:21:52.954 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-03-03T06:21:53.352 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-03-03T06:21:53.354 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-03-03T06:21:53.745 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-03-03T06:21:53.756 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-03-03T06:21:53.881 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-03-03T06:21:54.223 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-03-03T06:21:54.399 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-03-03T06:21:54.630 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-packaging (20.3-1) ... 2023-03-03T06:21:54.800 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-03-03T06:21:55.048 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-03-03T06:21:55.174 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-03-03T06:21:55.210 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-03-03T06:21:55.300 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-03-03T06:21:55.669 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-03-03T06:21:55.701 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-03-03T06:21:56.079 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-03-03T06:21:56.236 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-03-03T06:21:56.554 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-03-03T06:21:56.655 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2023-03-03T06:21:56.972 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-packaging (20.3-1) ... 2023-03-03T06:21:57.155 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-03-03T06:21:57.381 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-03-03T06:21:57.507 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-03-03T06:21:57.633 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-03-03T06:21:57.648 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up socat (1.7.3.3-2) ... 2023-03-03T06:21:57.775 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-ceph-common (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:21:58.034 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-03-03T06:21:58.226 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-03-03T06:21:58.377 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-03-03T06:21:58.578 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-03-03T06:21:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-03-03T06:21:59.004 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-03-03T06:21:59.012 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2023-03-03T06:21:59.130 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-03-03T06:21:59.531 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-03-03T06:21:59.555 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-03-03T06:21:59.966 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-03-03T06:22:00.057 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up socat (1.7.3.3-2) ... 2023-03-03T06:22:00.092 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-03-03T06:22:00.184 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-common (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:00.218 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-03-03T06:22:00.626 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-03-03T06:22:00.679 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-03-03T06:22:00.794 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-03-03T06:22:01.088 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-03-03T06:22:01.278 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-03-03T06:22:01.404 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-03-03T06:22:01.497 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-03-03T06:22:01.522 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-03-03T06:22:01.623 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-03-03T06:22:01.923 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-03-03T06:22:02.007 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-03-03T06:22:02.367 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-03-03T06:22:02.400 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-03-03T06:22:02.492 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-03-03T06:22:02.618 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-03-03T06:22:02.821 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-03-03T06:22:03.070 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-03-03T06:22:03.217 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-03-03T06:22:03.521 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-03-03T06:22:03.582 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-03-03T06:22:03.708 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-03-03T06:22:03.931 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-03-03T06:22:04.057 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-03-03T06:22:04.134 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-03-03T06:22:04.441 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-03-03T06:22:04.620 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-03-03T06:22:04.744 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-03-03T06:22:04.862 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-03-03T06:22:04.879 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-03-03T06:22:05.288 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-03-03T06:22:05.347 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-03-03T06:22:05.665 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-03-03T06:22:05.870 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-03-03T06:22:06.015 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-03-03T06:22:06.141 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-03-03T06:22:06.592 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-03-03T06:22:07.486 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-03-03T06:22:07.612 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-03-03T06:22:07.738 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-03-03T06:22:07.803 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-03-03T06:22:07.929 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-03-03T06:22:08.188 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-03-03T06:22:08.431 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-03-03T06:22:08.589 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-03-03T06:22:08.843 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-03-03T06:22:09.236 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-03-03T06:22:09.362 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-03-03T06:22:09.880 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-03-03T06:22:10.340 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2023-03-03T06:22:10.639 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-03-03T06:22:10.763 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-03-03T06:22:10.868 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-03-03T06:22:11.284 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-03-03T06:22:11.334 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-03-03T06:22:11.710 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-03-03T06:22:12.128 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-03-03T06:22:12.255 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-03-03T06:22:12.714 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-03-03T06:22:12.733 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-03-03T06:22:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-03-03T06:22:12.985 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-03-03T06:22:13.111 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rsa (4.0-3) ... 2023-03-03T06:22:13.167 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2023-03-03T06:22:13.529 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-03-03T06:22:13.655 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-03-03T06:22:13.691 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-03-03T06:22:14.089 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-03-03T06:22:14.172 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-03-03T06:22:14.515 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-03-03T06:22:14.958 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-03-03T06:22:15.460 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librados2 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:15.568 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-03-03T06:22:15.688 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2023-03-03T06:22:15.714 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-03-03T06:22:15.814 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librgw2 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:15.837 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-03-03T06:22:15.939 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libsqlite3-mod-ceph (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:15.979 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rsa (4.0-3) ... 2023-03-03T06:22:16.066 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libcephfs2 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:16.192 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-routes (2.4.1-2) ... 2023-03-03T06:22:16.397 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-03-03T06:22:16.523 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-03-03T06:22:16.593 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libradosstriper1 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:16.719 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-03-03T06:22:16.949 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-03-03T06:22:17.130 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librbd1 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:17.255 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-modules-core (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:17.359 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-03-03T06:22:17.484 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-fuse (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:17.773 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-03-03T06:22:17.774 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-03-03T06:22:17.802 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-03-03T06:22:18.320 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librados2 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:18.540 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2023-03-03T06:22:18.666 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librgw2 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:18.785 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libcephfs-dev (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:18.792 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libsqlite3-mod-ceph (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:18.918 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs2 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:18.920 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rados (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:19.044 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-routes (2.4.1-2) ... 2023-03-03T06:22:19.046 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up cephfs-top (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:19.172 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-03-03T06:22:19.462 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libradosstriper1 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:19.597 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-03-03T06:22:20.023 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librbd1 (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:20.151 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-modules-core (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:20.278 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-fuse (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:20.470 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rbd (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:20.570 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-03-03T06:22:20.571 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-03-03T06:22:20.596 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up rbd-fuse (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:20.722 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rgw (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:20.848 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cephfs (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:20.999 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up cephfs-shell (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:21.125 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-common (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:21.620 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs-dev (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:21.730 INFO:teuthology.orchestra.run.smithi003.stdout:Adding group ceph....done 2023-03-03T06:22:21.764 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rados (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:21.889 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-top (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:22.015 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-03-03T06:22:22.224 INFO:teuthology.orchestra.run.smithi003.stdout:Adding system user ceph....done 2023-03-03T06:22:22.469 INFO:teuthology.orchestra.run.smithi003.stdout:Setting system user ceph properties....done 2023-03-03T06:22:22.523 INFO:teuthology.orchestra.run.smithi003.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-03-03T06:22:22.750 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-03-03T06:22:23.331 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rbd (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:23.405 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-03-03T06:22:23.457 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up rbd-fuse (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:23.583 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rgw (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:23.717 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cephfs (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:23.851 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-shell (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:24.002 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-common (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:24.522 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up cephfs-mirror (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:24.601 INFO:teuthology.orchestra.run.smithi032.stdout:Adding group ceph....done 2023-03-03T06:22:24.649 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-test (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:24.808 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up radosgw (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:25.092 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user ceph....done 2023-03-03T06:22:25.159 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-03-03T06:22:25.159 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-03-03T06:22:25.336 INFO:teuthology.orchestra.run.smithi032.stdout:Setting system user ceph properties....done 2023-03-03T06:22:25.380 INFO:teuthology.orchestra.run.smithi032.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-03-03T06:22:25.614 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-03-03T06:22:26.225 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-03-03T06:22:26.759 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-base (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:27.370 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-03-03T06:22:27.431 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-mirror (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:27.599 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-test (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:27.725 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up radosgw (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:28.059 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-03-03T06:22:28.060 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-03-03T06:22:28.538 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mds (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:28.883 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-03-03T06:22:28.883 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-03-03T06:22:29.560 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-base (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:29.956 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:30.159 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-03-03T06:22:30.261 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-03-03T06:22:30.261 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-03-03T06:22:31.253 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mds (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:31.316 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-osd (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:31.578 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-03-03T06:22:31.578 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-03-03T06:22:32.306 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-03-03T06:22:32.306 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-03-03T06:22:32.764 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:33.063 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-03-03T06:22:33.063 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-03-03T06:22:33.359 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-k8sevents (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:33.485 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:33.644 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mon (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:33.926 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-03-03T06:22:33.928 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-03-03T06:22:34.065 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-osd (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:34.937 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-cephadm (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:35.000 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-03-03T06:22:35.000 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-03-03T06:22:35.097 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:35.223 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-dashboard (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:35.382 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-volume (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:35.509 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-03-03T06:22:35.677 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-03-03T06:22:36.024 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-k8sevents (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:36.150 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:36.237 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-03-03T06:22:36.343 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mon (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:36.634 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-03-03T06:22:36.635 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-03-03T06:22:37.677 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-cephadm (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:38.138 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:38.305 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-dashboard (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:38.464 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-volume (18.0.0-2739-g472aaef4-1focal) ... 2023-03-03T06:22:38.591 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-03-03T06:22:38.768 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-03-03T06:22:39.344 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-03-03T06:22:42.865 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-03-03T06:22:43.917 INFO:teuthology.orchestra.run.smithi003.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-03-03T06:22:43.927 DEBUG:teuthology.orchestra.run.smithi003:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-03-03T06:22:44.083 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2023-03-03T06:22:44.288 INFO:teuthology.orchestra.run.smithi003.stdout:Building dependency tree... 2023-03-03T06:22:44.289 INFO:teuthology.orchestra.run.smithi003.stdout:Reading state information... 2023-03-03T06:22:44.475 INFO:teuthology.orchestra.run.smithi003.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-03-03T06:22:44.476 INFO:teuthology.orchestra.run.smithi003.stdout:flex is already the newest version (2.6.4-6.2). 2023-03-03T06:22:44.476 INFO:teuthology.orchestra.run.smithi003.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-03-03T06:22:44.476 INFO:teuthology.orchestra.run.smithi003.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-03-03T06:22:44.476 INFO:teuthology.orchestra.run.smithi003.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-03-03T06:22:44.476 INFO:teuthology.orchestra.run.smithi003.stdout:util-linux is already the newest version (2.34-0.1ubuntu9.3). 2023-03-03T06:22:44.477 INFO:teuthology.orchestra.run.smithi003.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2023-03-03T06:22:44.477 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages were automatically installed and are no longer required: 2023-03-03T06:22:44.478 INFO:teuthology.orchestra.run.smithi003.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-03-03T06:22:44.478 INFO:teuthology.orchestra.run.smithi003.stdout:Use 'sudo apt autoremove' to remove them. 2023-03-03T06:22:44.478 INFO:teuthology.orchestra.run.smithi003.stdout:The following additional packages will be installed: 2023-03-03T06:22:44.478 INFO:teuthology.orchestra.run.smithi003.stdout: libbluetooth3 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 2023-03-03T06:22:44.478 INFO:teuthology.orchestra.run.smithi003.stdout: libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 modemmanager 2023-03-03T06:22:44.478 INFO:teuthology.orchestra.run.smithi003.stdout: network-manager-pptp postgresql-12 postgresql-client-12 2023-03-03T06:22:44.479 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-client-common ppp pptp-linux usb-modeswitch usb-modeswitch-data 2023-03-03T06:22:44.479 INFO:teuthology.orchestra.run.smithi003.stdout: wpasupplicant 2023-03-03T06:22:44.479 INFO:teuthology.orchestra.run.smithi003.stdout:Suggested packages: 2023-03-03T06:22:44.479 INFO:teuthology.orchestra.run.smithi003.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2023-03-03T06:22:44.480 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-doc-12 libjson-perl comgt wvdial wpagui libengine-pkcs11-openssl 2023-03-03T06:22:44.597 INFO:teuthology.orchestra.run.smithi003.stdout:The following NEW packages will be installed: 2023-03-03T06:22:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-03-03T06:22:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2023-03-03T06:22:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-03-03T06:22:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2023-03-03T06:22:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2023-03-03T06:22:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-03-03T06:22:44.599 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages will be upgraded: 2023-03-03T06:22:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: libssl1.1 2023-03-03T06:22:44.697 INFO:teuthology.orchestra.run.smithi003.stdout:1 upgraded, 31 newly installed, 0 to remove and 151 not upgraded. 2023-03-03T06:22:44.697 INFO:teuthology.orchestra.run.smithi003.stdout:Need to get 24.0 MB of archives. 2023-03-03T06:22:44.697 INFO:teuthology.orchestra.run.smithi003.stdout:After this operation, 79.9 MB of additional disk space will be used. 2023-03-03T06:22:44.698 INFO:teuthology.orchestra.run.smithi003.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.17 [1,322 kB] 2023-03-03T06:22:44.837 INFO:teuthology.orchestra.run.smithi003.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-03-03T06:22:44.839 INFO:teuthology.orchestra.run.smithi003.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-03-03T06:22:44.843 INFO:teuthology.orchestra.run.smithi003.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2023-03-03T06:22:44.846 INFO:teuthology.orchestra.run.smithi003.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-03-03T06:22:44.846 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-03-03T06:22:44.847 INFO:teuthology.orchestra.run.smithi003.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2023-03-03T06:22:44.849 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-03-03T06:22:44.852 INFO:teuthology.orchestra.run.smithi003.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-03-03T06:22:44.852 INFO:teuthology.orchestra.run.smithi003.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-03-03T06:22:44.856 INFO:teuthology.orchestra.run.smithi003.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-03-03T06:22:44.892 INFO:teuthology.orchestra.run.smithi003.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-03-03T06:22:44.999 INFO:teuthology.orchestra.run.smithi003.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.14-0ubuntu0.20.04.1 [116 kB] 2023-03-03T06:22:45.004 INFO:teuthology.orchestra.run.smithi003.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-03-03T06:22:45.030 INFO:teuthology.orchestra.run.smithi003.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-03-03T06:22:45.030 INFO:teuthology.orchestra.run.smithi003.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.17 [1,585 kB] 2023-03-03T06:22:45.056 INFO:teuthology.orchestra.run.smithi003.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-03-03T06:22:45.056 INFO:teuthology.orchestra.run.smithi003.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-03-03T06:22:45.070 INFO:teuthology.orchestra.run.smithi003.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-03-03T06:22:45.086 INFO:teuthology.orchestra.run.smithi003.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-03-03T06:22:45.108 INFO:teuthology.orchestra.run.smithi003.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-03-03T06:22:45.113 INFO:teuthology.orchestra.run.smithi003.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-03-03T06:22:45.149 INFO:teuthology.orchestra.run.smithi003.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-03-03T06:22:45.191 INFO:teuthology.orchestra.run.smithi003.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2023-03-03T06:22:45.202 INFO:teuthology.orchestra.run.smithi003.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.14-0ubuntu0.20.04.1 [1,053 kB] 2023-03-03T06:22:45.281 INFO:teuthology.orchestra.run.smithi003.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2023-03-03T06:22:45.284 INFO:teuthology.orchestra.run.smithi003.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.14-0ubuntu0.20.04.1 [13.5 MB] 2023-03-03T06:22:45.462 INFO:teuthology.orchestra.run.smithi003.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2023-03-03T06:22:45.462 INFO:teuthology.orchestra.run.smithi003.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2023-03-03T06:22:45.463 INFO:teuthology.orchestra.run.smithi003.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2023-03-03T06:22:45.463 INFO:teuthology.orchestra.run.smithi003.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-03-03T06:22:45.463 INFO:teuthology.orchestra.run.smithi003.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-03-03T06:22:45.964 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-03-03T06:22:46.130 INFO:teuthology.orchestra.run.smithi003.stdout:Preconfiguring packages ... 2023-03-03T06:22:46.435 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 24.0 MB in 1s (28.1 MB/s) 2023-03-03T06:22:46.830 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118876 files and directories currently installed.) 2023-03-03T06:22:46.833 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../00-libssl1.1_1.1.1f-1ubuntu2.17_amd64.deb ... 2023-03-03T06:22:46.919 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-03-03T06:22:46.930 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-03-03T06:22:47.014 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.17) over (1.1.1f-1ubuntu2.16) ... 2023-03-03T06:22:47.084 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2023-03-03T06:22:47.281 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2023-03-03T06:22:47.281 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2023-03-03T06:22:47.470 INFO:teuthology.orchestra.run.smithi032.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-03-03T06:22:47.471 INFO:teuthology.orchestra.run.smithi032.stdout:flex is already the newest version (2.6.4-6.2). 2023-03-03T06:22:47.471 INFO:teuthology.orchestra.run.smithi032.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-03-03T06:22:47.471 INFO:teuthology.orchestra.run.smithi032.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-03-03T06:22:47.471 INFO:teuthology.orchestra.run.smithi032.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-03-03T06:22:47.471 INFO:teuthology.orchestra.run.smithi032.stdout:util-linux is already the newest version (2.34-0.1ubuntu9.3). 2023-03-03T06:22:47.471 INFO:teuthology.orchestra.run.smithi032.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2023-03-03T06:22:47.472 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2023-03-03T06:22:47.472 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-03-03T06:22:47.472 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2023-03-03T06:22:47.473 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2023-03-03T06:22:47.473 INFO:teuthology.orchestra.run.smithi032.stdout: libbluetooth3 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 2023-03-03T06:22:47.473 INFO:teuthology.orchestra.run.smithi032.stdout: libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 modemmanager 2023-03-03T06:22:47.473 INFO:teuthology.orchestra.run.smithi032.stdout: network-manager-pptp postgresql-12 postgresql-client-12 2023-03-03T06:22:47.473 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-client-common ppp pptp-linux usb-modeswitch usb-modeswitch-data 2023-03-03T06:22:47.473 INFO:teuthology.orchestra.run.smithi032.stdout: wpasupplicant 2023-03-03T06:22:47.474 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2023-03-03T06:22:47.474 INFO:teuthology.orchestra.run.smithi032.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2023-03-03T06:22:47.474 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-doc-12 libjson-perl comgt wvdial wpagui libengine-pkcs11-openssl 2023-03-03T06:22:47.583 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2023-03-03T06:22:47.584 INFO:teuthology.orchestra.run.smithi032.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-03-03T06:22:47.584 INFO:teuthology.orchestra.run.smithi032.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2023-03-03T06:22:47.585 INFO:teuthology.orchestra.run.smithi032.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-03-03T06:22:47.585 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2023-03-03T06:22:47.585 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2023-03-03T06:22:47.585 INFO:teuthology.orchestra.run.smithi032.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-03-03T06:22:47.586 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2023-03-03T06:22:47.587 INFO:teuthology.orchestra.run.smithi032.stdout: libssl1.1 2023-03-03T06:22:47.609 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-03-03T06:22:47.631 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2_amd64.deb ... 2023-03-03T06:22:47.660 INFO:teuthology.orchestra.run.smithi032.stdout:1 upgraded, 31 newly installed, 0 to remove and 151 not upgraded. 2023-03-03T06:22:47.661 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 24.0 MB of archives. 2023-03-03T06:22:47.661 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 79.9 MB of additional disk space will be used. 2023-03-03T06:22:47.661 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.17 [1,322 kB] 2023-03-03T06:22:47.676 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-03-03T06:22:47.801 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-03-03T06:22:47.803 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-03-03T06:22:47.808 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2023-03-03T06:22:47.812 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-03-03T06:22:47.812 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-03-03T06:22:47.813 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2023-03-03T06:22:47.814 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-03-03T06:22:47.817 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-03-03T06:22:47.817 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-03-03T06:22:47.820 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-03-03T06:22:47.856 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-03-03T06:22:47.964 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.14-0ubuntu0.20.04.1 [116 kB] 2023-03-03T06:22:47.969 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-03-03T06:22:47.995 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-03-03T06:22:47.996 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.17 [1,585 kB] 2023-03-03T06:22:48.021 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-03-03T06:22:48.022 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-03-03T06:22:48.033 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-03-03T06:22:48.048 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-03-03T06:22:48.070 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-03-03T06:22:48.074 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-03-03T06:22:48.110 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-03-03T06:22:48.161 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2023-03-03T06:22:48.174 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.14-0ubuntu0.20.04.1 [1,053 kB] 2023-03-03T06:22:48.189 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package dump. 2023-03-03T06:22:48.211 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../02-dump_0.4b46-6_amd64.deb ... 2023-03-03T06:22:48.255 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking dump (0.4b46-6) ... 2023-03-03T06:22:48.257 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2023-03-03T06:22:48.261 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.14-0ubuntu0.20.04.1 [13.5 MB] 2023-03-03T06:22:48.423 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2023-03-03T06:22:48.423 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2023-03-03T06:22:48.424 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2023-03-03T06:22:48.424 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-03-03T06:22:48.424 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-03-03T06:22:48.659 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package indent. 2023-03-03T06:22:48.681 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../03-indent_2.2.12-1_amd64.deb ... 2023-03-03T06:22:48.725 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking indent (2.2.12-1) ... 2023-03-03T06:22:49.095 INFO:teuthology.orchestra.run.smithi032.stdout:Preconfiguring packages ... 2023-03-03T06:22:49.103 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-03-03T06:22:49.126 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-5_amd64.deb ... 2023-03-03T06:22:49.170 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-03-03T06:22:49.404 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 24.0 MB in 1s (28.9 MB/s) 2023-03-03T06:22:49.565 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-03-03T06:22:49.587 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../05-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-03-03T06:22:49.631 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-03-03T06:22:49.777 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118876 files and directories currently installed.) 2023-03-03T06:22:49.780 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../00-libssl1.1_1.1.1f-1ubuntu2.17_amd64.deb ... 2023-03-03T06:22:49.959 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-03-03T06:22:49.979 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.17) over (1.1.1f-1ubuntu2.16) ... 2023-03-03T06:22:49.982 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../06-libelf-dev_0.176-1.1build1_amd64.deb ... 2023-03-03T06:22:50.025 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2023-03-03T06:22:50.445 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-03-03T06:22:50.467 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../07-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-03-03T06:22:50.511 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-03-03T06:22:50.583 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-03-03T06:22:50.605 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2_amd64.deb ... 2023-03-03T06:22:50.666 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-03-03T06:22:50.839 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libmbim-proxy. 2023-03-03T06:22:50.861 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../08-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-03-03T06:22:50.905 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-03-03T06:22:51.077 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package dump. 2023-03-03T06:22:51.099 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../02-dump_0.4b46-6_amd64.deb ... 2023-03-03T06:22:51.143 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking dump (0.4b46-6) ... 2023-03-03T06:22:51.283 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-03-03T06:22:51.306 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../09-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-03-03T06:22:51.349 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-03-03T06:22:51.548 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package indent. 2023-03-03T06:22:51.570 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../03-indent_2.2.12-1_amd64.deb ... 2023-03-03T06:22:51.614 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking indent (2.2.12-1) ... 2023-03-03T06:22:51.786 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libndp0:amd64. 2023-03-03T06:22:51.808 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../10-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-03-03T06:22:51.852 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-03-03T06:22:51.992 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-03-03T06:22:52.012 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-5_amd64.deb ... 2023-03-03T06:22:52.049 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-03-03T06:22:52.233 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libnm0:amd64. 2023-03-03T06:22:52.256 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../11-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-03-03T06:22:52.299 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-03-03T06:22:52.444 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-03-03T06:22:52.468 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../05-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-03-03T06:22:52.511 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-03-03T06:22:52.752 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libpq5:amd64. 2023-03-03T06:22:52.774 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../12-libpq5_12.14-0ubuntu0.20.04.1_amd64.deb ... 2023-03-03T06:22:52.819 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libpq5:amd64 (12.14-0ubuntu0.20.04.1) ... 2023-03-03T06:22:52.864 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-03-03T06:22:52.886 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../06-libelf-dev_0.176-1.1build1_amd64.deb ... 2023-03-03T06:22:52.930 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2023-03-03T06:22:53.315 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-03-03T06:22:53.337 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../13-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-03-03T06:22:53.382 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-03-03T06:22:53.618 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-03-03T06:22:53.641 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../07-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-03-03T06:22:53.684 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-03-03T06:22:53.793 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqmi-proxy. 2023-03-03T06:22:53.815 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../14-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-03-03T06:22:53.859 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-03-03T06:22:54.020 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libmbim-proxy. 2023-03-03T06:22:54.042 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../08-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-03-03T06:22:54.087 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-03-03T06:22:54.186 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-03-03T06:22:54.208 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../15-libssl-dev_1.1.1f-1ubuntu2.17_amd64.deb ... 2023-03-03T06:22:54.253 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.17) ... 2023-03-03T06:22:54.473 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-03-03T06:22:54.494 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../09-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-03-03T06:22:54.539 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-03-03T06:22:54.834 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-03-03T06:22:54.854 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../16-libteamdctl0_1.30-1_amd64.deb ... 2023-03-03T06:22:54.900 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-03-03T06:22:54.953 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libndp0:amd64. 2023-03-03T06:22:54.976 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../10-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-03-03T06:22:55.019 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-03-03T06:22:55.328 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package modemmanager. 2023-03-03T06:22:55.350 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../17-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-03-03T06:22:55.397 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libnm0:amd64. 2023-03-03T06:22:55.419 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../11-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-03-03T06:22:55.430 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-03-03T06:22:55.464 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-03-03T06:22:55.891 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libpq5:amd64. 2023-03-03T06:22:55.913 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../12-libpq5_12.14-0ubuntu0.20.04.1_amd64.deb ... 2023-03-03T06:22:55.957 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libpq5:amd64 (12.14-0ubuntu0.20.04.1) ... 2023-03-03T06:22:56.192 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package wpasupplicant. 2023-03-03T06:22:56.214 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../18-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-03-03T06:22:56.258 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-03-03T06:22:56.377 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-03-03T06:22:56.399 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../13-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-03-03T06:22:56.443 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-03-03T06:22:56.854 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqmi-proxy. 2023-03-03T06:22:56.874 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package network-manager. 2023-03-03T06:22:56.876 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../14-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-03-03T06:22:56.895 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../19-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-03-03T06:22:56.946 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-03-03T06:22:57.012 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-03-03T06:22:57.265 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-03-03T06:22:57.287 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../15-libssl-dev_1.1.1f-1ubuntu2.17_amd64.deb ... 2023-03-03T06:22:57.331 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.17) ... 2023-03-03T06:22:57.864 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ppp. 2023-03-03T06:22:57.886 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../20-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-03-03T06:22:57.928 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-03-03T06:22:57.950 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../16-libteamdctl0_1.30-1_amd64.deb ... 2023-03-03T06:22:58.004 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-03-03T06:22:58.465 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package modemmanager. 2023-03-03T06:22:58.480 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-03-03T06:22:58.487 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../17-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-03-03T06:22:58.566 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-03-03T06:22:58.868 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package pptp-linux. 2023-03-03T06:22:58.890 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../21-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-03-03T06:22:58.934 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-03-03T06:22:59.295 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package wpasupplicant. 2023-03-03T06:22:59.316 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../18-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-03-03T06:22:59.361 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-03-03T06:22:59.405 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package network-manager-pptp. 2023-03-03T06:22:59.426 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../22-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-03-03T06:22:59.532 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-03-03T06:22:59.874 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-client-common. 2023-03-03T06:22:59.889 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../23-postgresql-client-common_214ubuntu0.1_all.deb ... 2023-03-03T06:22:59.933 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2023-03-03T06:23:00.026 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package network-manager. 2023-03-03T06:23:00.047 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../19-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-03-03T06:23:00.165 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-03-03T06:23:00.311 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-client-12. 2023-03-03T06:23:00.326 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../24-postgresql-client-12_12.14-0ubuntu0.20.04.1_amd64.deb ... 2023-03-03T06:23:00.369 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-client-12 (12.14-0ubuntu0.20.04.1) ... 2023-03-03T06:23:01.049 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ppp. 2023-03-03T06:23:01.057 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-common. 2023-03-03T06:23:01.071 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../20-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-03-03T06:23:01.072 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../25-postgresql-common_214ubuntu0.1_all.deb ... 2023-03-03T06:23:01.158 INFO:teuthology.orchestra.run.smithi003.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-03-03T06:23:01.187 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2023-03-03T06:23:01.650 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-03-03T06:23:01.698 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-12. 2023-03-03T06:23:01.719 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../26-postgresql-12_12.14-0ubuntu0.20.04.1_amd64.deb ... 2023-03-03T06:23:01.780 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-12 (12.14-0ubuntu0.20.04.1) ... 2023-03-03T06:23:02.104 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package pptp-linux. 2023-03-03T06:23:02.125 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../21-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-03-03T06:23:02.179 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-03-03T06:23:02.632 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package network-manager-pptp. 2023-03-03T06:23:02.652 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../22-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-03-03T06:23:02.746 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-03-03T06:23:03.101 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-client-common. 2023-03-03T06:23:03.123 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../23-postgresql-client-common_214ubuntu0.1_all.deb ... 2023-03-03T06:23:03.167 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2023-03-03T06:23:03.475 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql. 2023-03-03T06:23:03.498 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../27-postgresql_12+214ubuntu0.1_all.deb ... 2023-03-03T06:23:03.542 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2023-03-03T06:23:03.580 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-client-12. 2023-03-03T06:23:03.600 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../24-postgresql-client-12_12.14-0ubuntu0.20.04.1_amd64.deb ... 2023-03-03T06:23:03.637 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-client-12 (12.14-0ubuntu0.20.04.1) ... 2023-03-03T06:23:03.852 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-client. 2023-03-03T06:23:03.874 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../28-postgresql-client_12+214ubuntu0.1_all.deb ... 2023-03-03T06:23:03.919 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2023-03-03T06:23:04.221 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-contrib. 2023-03-03T06:23:04.243 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../29-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2023-03-03T06:23:04.287 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2023-03-03T06:23:04.334 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-common. 2023-03-03T06:23:04.356 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../25-postgresql-common_214ubuntu0.1_all.deb ... 2023-03-03T06:23:04.445 INFO:teuthology.orchestra.run.smithi032.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-03-03T06:23:04.474 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2023-03-03T06:23:04.607 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-03-03T06:23:04.627 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../30-usb-modeswitch-data_20191128-3_all.deb ... 2023-03-03T06:23:04.664 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-03-03T06:23:05.015 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-12. 2023-03-03T06:23:05.036 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../26-postgresql-12_12.14-0ubuntu0.20.04.1_amd64.deb ... 2023-03-03T06:23:05.095 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-12 (12.14-0ubuntu0.20.04.1) ... 2023-03-03T06:23:05.126 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package usb-modeswitch. 2023-03-03T06:23:05.148 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../31-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-03-03T06:23:05.227 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-03-03T06:23:05.600 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2023-03-03T06:23:05.815 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.17) ... 2023-03-03T06:23:06.233 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libpq5:amd64 (12.14-0ubuntu0.20.04.1) ... 2023-03-03T06:23:06.359 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-03-03T06:23:06.485 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-03-03T06:23:06.611 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-03-03T06:23:06.817 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql. 2023-03-03T06:23:06.833 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../27-postgresql_12+214ubuntu0.1_all.deb ... 2023-03-03T06:23:06.875 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2023-03-03T06:23:07.074 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-03-03T06:23:07.487 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-03-03T06:23:07.488 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-client. 2023-03-03T06:23:07.488 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../28-postgresql-client_12+214ubuntu0.1_all.deb ... 2023-03-03T06:23:07.488 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2023-03-03T06:23:07.571 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-contrib. 2023-03-03T06:23:07.592 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../29-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2023-03-03T06:23:07.638 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2023-03-03T06:23:07.949 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-03-03T06:23:07.971 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../30-usb-modeswitch-data_20191128-3_all.deb ... 2023-03-03T06:23:08.015 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-03-03T06:23:08.189 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-client-12 (12.14-0ubuntu0.20.04.1) ... 2023-03-03T06:23:08.493 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package usb-modeswitch. 2023-03-03T06:23:08.516 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../31-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-03-03T06:23:08.589 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-03-03T06:23:09.311 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2023-03-03T06:23:09.523 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.17) ... 2023-03-03T06:23:09.925 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libpq5:amd64 (12.14-0ubuntu0.20.04.1) ... 2023-03-03T06:23:10.059 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-03-03T06:23:10.185 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-03-03T06:23:10.311 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-03-03T06:23:10.784 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-03-03T06:23:10.784 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-03-03T06:23:11.267 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-03-03T06:23:11.385 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-03-03T06:23:11.520 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2023-03-03T06:23:11.880 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-client-12 (12.14-0ubuntu0.20.04.1) ... 2023-03-03T06:23:12.903 INFO:teuthology.orchestra.run.smithi003.stdout:Adding user postgres to group ssl-cert 2023-03-03T06:23:13.322 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:13.323 INFO:teuthology.orchestra.run.smithi003.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-03-03T06:23:13.965 INFO:teuthology.orchestra.run.smithi003.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-03-03T06:23:13.965 INFO:teuthology.orchestra.run.smithi003.stdout:Removing obsolete dictionary files: 2023-03-03T06:23:14.607 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-03-03T06:23:14.978 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-03-03T06:23:15.093 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-03-03T06:23:15.236 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2023-03-03T06:23:16.347 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.17) ... 2023-03-03T06:23:16.472 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-03-03T06:23:16.598 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2023-03-03T06:23:16.625 INFO:teuthology.orchestra.run.smithi032.stdout:Adding user postgres to group ssl-cert 2023-03-03T06:23:16.725 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-03-03T06:23:16.850 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-03-03T06:23:17.071 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:17.072 INFO:teuthology.orchestra.run.smithi032.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-03-03T06:23:17.756 INFO:teuthology.orchestra.run.smithi032.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-03-03T06:23:17.756 INFO:teuthology.orchestra.run.smithi032.stdout:Removing obsolete dictionary files: 2023-03-03T06:23:18.205 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-03-03T06:23:18.431 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-03-03T06:23:18.706 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up dump (0.4b46-6) ... 2023-03-03T06:23:18.827 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-03-03T06:23:18.891 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-03-03T06:23:19.017 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-03-03T06:23:19.143 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up indent (2.2.12-1) ... 2023-03-03T06:23:19.269 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-03-03T06:23:19.462 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-03-03T06:23:19.588 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-12 (12.14-0ubuntu0.20.04.1) ... 2023-03-03T06:23:20.154 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.17) ... 2023-03-03T06:23:20.164 INFO:teuthology.orchestra.run.smithi003.stdout:Creating new PostgreSQL cluster 12/main ... 2023-03-03T06:23:20.211 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2023-03-03T06:23:20.280 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-03-03T06:23:20.295 INFO:teuthology.orchestra.run.smithi003.stdout:The files belonging to this database system will be owned by user "postgres". 2023-03-03T06:23:20.295 INFO:teuthology.orchestra.run.smithi003.stdout:This user must also own the server process. 2023-03-03T06:23:20.296 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:20.296 INFO:teuthology.orchestra.run.smithi003.stdout:The database cluster will be initialized with locale "en_US". 2023-03-03T06:23:20.296 INFO:teuthology.orchestra.run.smithi003.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-03-03T06:23:20.296 INFO:teuthology.orchestra.run.smithi003.stdout:The default text search configuration will be set to "english". 2023-03-03T06:23:20.296 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:20.297 INFO:teuthology.orchestra.run.smithi003.stdout:Data page checksums are disabled. 2023-03-03T06:23:20.297 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:20.297 INFO:teuthology.orchestra.run.smithi003.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2023-03-03T06:23:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:creating subdirectories ... ok 2023-03-03T06:23:20.300 INFO:teuthology.orchestra.run.smithi003.stdout:selecting dynamic shared memory implementation ... posix 2023-03-03T06:23:20.349 INFO:teuthology.orchestra.run.smithi003.stdout:selecting default max_connections ... 100 2023-03-03T06:23:20.405 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2023-03-03T06:23:20.418 INFO:teuthology.orchestra.run.smithi003.stdout:selecting default shared_buffers ... 128MB 2023-03-03T06:23:20.446 INFO:teuthology.orchestra.run.smithi003.stdout:selecting default time zone ... Etc/UTC 2023-03-03T06:23:20.447 INFO:teuthology.orchestra.run.smithi003.stdout:creating configuration files ... ok 2023-03-03T06:23:20.531 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-03-03T06:23:20.657 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-03-03T06:23:20.699 INFO:teuthology.orchestra.run.smithi003.stdout:running bootstrap script ... ok 2023-03-03T06:23:21.429 INFO:teuthology.orchestra.run.smithi003.stdout:performing post-bootstrap initialization ... ok 2023-03-03T06:23:21.945 INFO:teuthology.orchestra.run.smithi003.stdout:syncing data to disk ... ok 2023-03-03T06:23:21.946 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:21.946 INFO:teuthology.orchestra.run.smithi003.stdout:Success. You can now start the database server using: 2023-03-03T06:23:21.946 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:21.946 INFO:teuthology.orchestra.run.smithi003.stdout: pg_ctlcluster 12 main start 2023-03-03T06:23:21.946 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:22.013 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-03-03T06:23:22.504 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up dump (0.4b46-6) ... 2023-03-03T06:23:22.625 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-03-03T06:23:22.658 INFO:teuthology.orchestra.run.smithi003.stdout:Ver Cluster Port Status Owner Data directory Log file 2023-03-03T06:23:22.658 INFO:teuthology.orchestra.run.smithi003.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2023-03-03T06:23:22.689 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-03-03T06:23:22.815 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-03-03T06:23:22.949 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up indent (2.2.12-1) ... 2023-03-03T06:23:22.961 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-03-03T06:23:23.075 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-03-03T06:23:23.268 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-03-03T06:23:23.444 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-12 (12.14-0ubuntu0.20.04.1) ... 2023-03-03T06:23:24.038 INFO:teuthology.orchestra.run.smithi032.stdout:Creating new PostgreSQL cluster 12/main ... 2023-03-03T06:23:24.083 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2023-03-03T06:23:24.162 INFO:teuthology.orchestra.run.smithi032.stdout:The files belonging to this database system will be owned by user "postgres". 2023-03-03T06:23:24.162 INFO:teuthology.orchestra.run.smithi032.stdout:This user must also own the server process. 2023-03-03T06:23:24.163 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:24.163 INFO:teuthology.orchestra.run.smithi032.stdout:The database cluster will be initialized with locale "en_US". 2023-03-03T06:23:24.163 INFO:teuthology.orchestra.run.smithi032.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-03-03T06:23:24.163 INFO:teuthology.orchestra.run.smithi032.stdout:The default text search configuration will be set to "english". 2023-03-03T06:23:24.163 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:24.163 INFO:teuthology.orchestra.run.smithi032.stdout:Data page checksums are disabled. 2023-03-03T06:23:24.163 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:24.164 INFO:teuthology.orchestra.run.smithi032.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2023-03-03T06:23:24.167 INFO:teuthology.orchestra.run.smithi032.stdout:creating subdirectories ... ok 2023-03-03T06:23:24.168 INFO:teuthology.orchestra.run.smithi032.stdout:selecting dynamic shared memory implementation ... posix 2023-03-03T06:23:24.214 INFO:teuthology.orchestra.run.smithi032.stdout:selecting default max_connections ... 100 2023-03-03T06:23:24.279 INFO:teuthology.orchestra.run.smithi032.stdout:selecting default shared_buffers ... 128MB 2023-03-03T06:23:24.308 INFO:teuthology.orchestra.run.smithi032.stdout:selecting default time zone ... Etc/UTC 2023-03-03T06:23:24.309 INFO:teuthology.orchestra.run.smithi032.stdout:creating configuration files ... ok 2023-03-03T06:23:24.560 INFO:teuthology.orchestra.run.smithi032.stdout:running bootstrap script ... ok 2023-03-03T06:23:25.307 INFO:teuthology.orchestra.run.smithi032.stdout:performing post-bootstrap initialization ... ok 2023-03-03T06:23:25.843 INFO:teuthology.orchestra.run.smithi032.stdout:syncing data to disk ... ok 2023-03-03T06:23:25.843 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:25.844 INFO:teuthology.orchestra.run.smithi032.stdout:Success. You can now start the database server using: 2023-03-03T06:23:25.844 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:25.844 INFO:teuthology.orchestra.run.smithi032.stdout: pg_ctlcluster 12 main start 2023-03-03T06:23:25.844 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:26.556 INFO:teuthology.orchestra.run.smithi032.stdout:Ver Cluster Port Status Owner Data directory Log file 2023-03-03T06:23:26.556 INFO:teuthology.orchestra.run.smithi032.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2023-03-03T06:23:26.881 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-03-03T06:23:26.962 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-03-03T06:23:27.088 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-03-03T06:23:27.256 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2023-03-03T06:23:27.507 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-03-03T06:23:28.902 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:28.903 INFO:teuthology.orchestra.run.smithi003.stdout:The following network interfaces were found in /etc/network/interfaces 2023-03-03T06:23:28.903 INFO:teuthology.orchestra.run.smithi003.stdout:which means they are currently configured by ifupdown: 2023-03-03T06:23:28.903 INFO:teuthology.orchestra.run.smithi003.stdout:- ens1f0 2023-03-03T06:23:28.903 INFO:teuthology.orchestra.run.smithi003.stdout:If you want to manage those interfaces with NetworkManager instead 2023-03-03T06:23:28.903 INFO:teuthology.orchestra.run.smithi003.stdout:remove their configuration from /etc/network/interfaces. 2023-03-03T06:23:28.903 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:29.172 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-03-03T06:23:29.857 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-03-03T06:23:30.854 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-03-03T06:23:30.988 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-03-03T06:23:31.004 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-03-03T06:23:31.156 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2023-03-03T06:23:31.282 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-03-03T06:23:32.127 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-03-03T06:23:32.261 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-03-03T06:23:32.446 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2023-03-03T06:23:32.572 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2023-03-03T06:23:32.642 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:32.642 INFO:teuthology.orchestra.run.smithi032.stdout:The following network interfaces were found in /etc/network/interfaces 2023-03-03T06:23:32.642 INFO:teuthology.orchestra.run.smithi032.stdout:which means they are currently configured by ifupdown: 2023-03-03T06:23:32.642 INFO:teuthology.orchestra.run.smithi032.stdout:- ens1f0 2023-03-03T06:23:32.642 INFO:teuthology.orchestra.run.smithi032.stdout:If you want to manage those interfaces with NetworkManager instead 2023-03-03T06:23:32.643 INFO:teuthology.orchestra.run.smithi032.stdout:remove their configuration from /etc/network/interfaces. 2023-03-03T06:23:32.643 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:32.698 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-03-03T06:23:32.824 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-03-03T06:23:32.916 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-03-03T06:23:32.950 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-03-03T06:23:33.303 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-03-03T06:23:33.303 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-03-03T06:23:33.587 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-03-03T06:23:34.486 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-03-03T06:23:34.723 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-03-03T06:23:34.856 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-03-03T06:23:35.023 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-03-03T06:23:35.741 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-03-03T06:23:35.910 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-03-03T06:23:36.045 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-03-03T06:23:36.221 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2023-03-03T06:23:36.364 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2023-03-03T06:23:36.490 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-03-03T06:23:36.616 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-03-03T06:23:36.750 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-03-03T06:23:37.103 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-03-03T06:23:37.104 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-03-03T06:23:38.311 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-03-03T06:23:38.688 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-03-03T06:23:38.873 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-03-03T06:23:39.456 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-03-03T06:23:50.049 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-03-03T06:23:50.169 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-03-03T06:23:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-03-03T06:23:51.733 DEBUG:teuthology.parallel:result is None 2023-03-03T06:23:53.535 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-03-03T06:23:53.660 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-03-03T06:23:55.195 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-03-03T06:23:55.201 DEBUG:teuthology.parallel:result is None 2023-03-03T06:23:55.202 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=472aaef4ec3b3a2c29f73ca67854785a910f1f9b 2023-03-03T06:23:55.340 DEBUG:teuthology.orchestra.run.smithi003:> dpkg-query -W -f '${Version}' ceph 2023-03-03T06:23:55.384 INFO:teuthology.orchestra.run.smithi003.stdout:18.0.0-2739-g472aaef4-1focal 2023-03-03T06:23:55.385 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2739-g472aaef4-1focal 2023-03-03T06:23:55.385 INFO:teuthology.task.install:The correct ceph version 18.0.0-2739-g472aaef4-1focal is installed. 2023-03-03T06:23:55.386 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=472aaef4ec3b3a2c29f73ca67854785a910f1f9b 2023-03-03T06:23:55.521 DEBUG:teuthology.orchestra.run.smithi032:> dpkg-query -W -f '${Version}' ceph 2023-03-03T06:23:55.564 INFO:teuthology.orchestra.run.smithi032.stdout:18.0.0-2739-g472aaef4-1focal 2023-03-03T06:23:55.564 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2739-g472aaef4-1focal 2023-03-03T06:23:55.565 INFO:teuthology.task.install:The correct ceph version 18.0.0-2739-g472aaef4-1focal is installed. 2023-03-03T06:23:55.565 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-03-03T06:23:55.566 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:23:55.566 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-03T06:23:55.613 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:23:55.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-03T06:23:55.657 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-03-03T06:23:55.657 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:23:55.658 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2023-03-03T06:23:55.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-03T06:23:55.788 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:23:55.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2023-03-03T06:23:55.834 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-03T06:23:55.918 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-03-03T06:23:55.918 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:23:55.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-03T06:23:55.964 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-03T06:23:56.050 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:23:56.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-03T06:23:56.094 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-03T06:23:56.183 INFO:teuthology.run_tasks:Running task ceph... 2023-03-03T06:23:56.317 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-03-03T06:23:56.318 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /var/log/ceph 2023-03-03T06:23:56.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /var/log/ceph 2023-03-03T06:23:56.368 INFO:tasks.ceph:Disabling ceph logrotate... 2023-03-03T06:23:56.369 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-03T06:23:56.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-03T06:23:56.457 INFO:tasks.ceph:Creating extra log directories... 2023-03-03T06:23:56.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-03T06:23:56.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-03T06:23:56.593 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-03-03T06:23:56.593 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-03-03T06:23:56.593 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-03-03T06:23:56.594 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:23:56.594 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-03-03T06:23:56.617 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-03-03T06:23:56.776 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:23:56.777 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-03-03T06:23:56.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-03-03T06:23:56.961 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-03-03T06:23:56.961 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-03-03T06:23:56.962 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/7192143', 'branch': 'wip-vshankar-testing-20230303.023823', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7192143', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '75534f74ac3dd06f86697ff862e8be2945b3c8e6'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20230303.023823'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '472aaef4ec3b3a2c29f73ca67854785a910f1f9b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '472aaef4ec3b3a2c29f73ca67854785a910f1f9b'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20230303.023823', 'sha1': '472aaef4ec3b3a2c29f73ca67854785a910f1f9b'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '472aaef4ec3b3a2c29f73ca67854785a910f1f9b', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20230303.023823', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_472aaef4ec3b3a2c29f73ca67854785a910f1f9b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '472aaef4ec3b3a2c29f73ca67854785a910f1f9b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi003.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0=', 'smithi032.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '75534f74ac3dd06f86697ff862e8be2945b3c8e6'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '472aaef4ec3b3a2c29f73ca67854785a910f1f9b'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6', 'timestamp': '2023-03-03_04:39:14', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.31660'} 2023-03-03T06:23:56.962 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-03T06:23:56.971 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-03T06:23:56.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/run/ceph 2023-03-03T06:23:57.041 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/run/ceph 2023-03-03T06:23:57.089 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:23:57.090 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2023-03-03T06:23:57.143 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-03T06:23:57.143 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2023-03-03T06:23:57.208 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-03T06:23:57.208 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-03T06:23:57.208 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 498 Links: 1 2023-03-03T06:23:57.208 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-03T06:23:57.209 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-03-03 06:18:11.888291671 +0000 2023-03-03T06:23:57.209 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-03-03 06:18:11.332272796 +0000 2023-03-03T06:23:57.209 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-03-03 06:18:11.332272796 +0000 2023-03-03T06:23:57.209 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-03-03T06:23:57.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-03T06:23:57.292 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-03-03T06:23:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-03-03T06:23:57.293 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000605352 s, 846 kB/s 2023-03-03T06:23:57.295 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-03T06:23:57.371 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2023-03-03T06:23:57.435 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-03T06:23:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-03T06:23:57.436 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 504 Links: 1 2023-03-03T06:23:57.436 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-03T06:23:57.436 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-03-03 06:18:12.788320661 +0000 2023-03-03T06:23:57.436 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-03-03 06:18:12.232302895 +0000 2023-03-03T06:23:57.436 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-03-03 06:18:12.232302895 +0000 2023-03-03T06:23:57.436 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-03-03T06:23:57.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-03T06:23:57.519 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-03-03T06:23:57.519 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-03-03T06:23:57.519 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000711131 s, 720 kB/s 2023-03-03T06:23:57.523 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-03T06:23:57.599 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2023-03-03T06:23:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-03T06:23:57.663 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-03T06:23:57.663 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 511 Links: 1 2023-03-03T06:23:57.664 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-03T06:23:57.664 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-03-03 06:18:13.676347689 +0000 2023-03-03T06:23:57.664 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-03-03 06:18:13.140331634 +0000 2023-03-03T06:23:57.664 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-03-03 06:18:13.140331634 +0000 2023-03-03T06:23:57.664 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-03-03T06:23:57.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-03T06:23:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-03-03T06:23:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-03-03T06:23:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.00087309 s, 586 kB/s 2023-03-03T06:23:57.752 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-03T06:23:57.826 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2023-03-03T06:23:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-03T06:23:57.897 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-03T06:23:57.898 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 516 Links: 1 2023-03-03T06:23:57.898 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-03T06:23:57.898 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2023-03-03 06:18:14.616375230 +0000 2023-03-03T06:23:57.898 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2023-03-03 06:18:14.048358786 +0000 2023-03-03T06:23:57.898 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2023-03-03 06:18:14.048358786 +0000 2023-03-03T06:23:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2023-03-03T06:23:57.899 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-03T06:23:57.979 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2023-03-03T06:23:57.980 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2023-03-03T06:23:57.980 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000694378 s, 737 kB/s 2023-03-03T06:23:57.984 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-03T06:23:58.061 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-03T06:23:58.062 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:23:58.062 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2023-03-03T06:23:58.084 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-03T06:23:58.085 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2023-03-03T06:23:58.149 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-03T06:23:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-03T06:23:58.150 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 500 Links: 1 2023-03-03T06:23:58.150 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-03T06:23:58.150 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-03-03 06:18:11.626753343 +0000 2023-03-03T06:23:58.150 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-03-03 06:18:11.042766783 +0000 2023-03-03T06:23:58.150 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-03-03 06:18:11.042766783 +0000 2023-03-03T06:23:58.150 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-03-03T06:23:58.151 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-03T06:23:58.233 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-03-03T06:23:58.233 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-03-03T06:23:58.234 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000480336 s, 1.1 MB/s 2023-03-03T06:23:58.237 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-03T06:23:58.315 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2023-03-03T06:23:58.380 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-03T06:23:58.380 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-03T06:23:58.380 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 506 Links: 1 2023-03-03T06:23:58.381 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-03T06:23:58.381 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-03-03 06:18:11.950745888 +0000 2023-03-03T06:23:58.381 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-03-03 06:18:11.946745979 +0000 2023-03-03T06:23:58.381 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-03-03 06:18:11.946745979 +0000 2023-03-03T06:23:58.381 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-03-03T06:23:58.382 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-03T06:23:58.462 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-03-03T06:23:58.463 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-03-03T06:23:58.463 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000596075 s, 859 kB/s 2023-03-03T06:23:58.468 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-03T06:23:58.547 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2023-03-03T06:23:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-03T06:23:58.613 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-03T06:23:58.613 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 510 Links: 1 2023-03-03T06:23:58.613 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-03T06:23:58.614 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-03-03 06:18:13.398712565 +0000 2023-03-03T06:23:58.614 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-03-03 06:18:12.818725914 +0000 2023-03-03T06:23:58.614 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-03-03 06:18:12.818725914 +0000 2023-03-03T06:23:58.614 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-03-03T06:23:58.615 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-03T06:23:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-03-03T06:23:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-03-03T06:23:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000737368 s, 694 kB/s 2023-03-03T06:23:58.704 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-03T06:23:58.778 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2023-03-03T06:23:58.840 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-03T06:23:58.841 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-03T06:23:58.841 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 516 Links: 1 2023-03-03T06:23:58.842 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-03T06:23:58.842 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-03-03 06:18:13.718705201 +0000 2023-03-03T06:23:58.842 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-03-03 06:18:13.714705294 +0000 2023-03-03T06:23:58.842 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-03-03 06:18:13.714705294 +0000 2023-03-03T06:23:58.842 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2023-03-03T06:23:58.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-03T06:23:58.925 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-03-03T06:23:58.926 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-03-03T06:23:58.926 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000547035 s, 936 kB/s 2023-03-03T06:23:58.930 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-03T06:23:59.002 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-03T06:23:59.002 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-03-03T06:23:59.003 INFO:tasks.ceph:Generating config... 2023-03-03T06:23:59.004 INFO:tasks.ceph:[client] client mount timeout = 600 2023-03-03T06:23:59.004 INFO:tasks.ceph:[client] debug client = 20 2023-03-03T06:23:59.005 INFO:tasks.ceph:[client] debug ms = 1 2023-03-03T06:23:59.005 INFO:tasks.ceph:[client] pid file = 2023-03-03T06:23:59.005 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-03-03T06:23:59.006 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-03-03T06:23:59.006 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-03-03T06:23:59.006 INFO:tasks.ceph:[mds] debug mds = 20 2023-03-03T06:23:59.006 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-03-03T06:23:59.006 INFO:tasks.ceph:[mds] debug ms = 1 2023-03-03T06:23:59.007 INFO:tasks.ceph:[mds] mds debug frag = True 2023-03-03T06:23:59.007 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-03-03T06:23:59.007 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-03-03T06:23:59.007 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-03-03T06:23:59.007 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-03-03T06:23:59.007 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-03-03T06:23:59.008 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-03-03T06:23:59.009 INFO:tasks.ceph:[mgr] debug client = 20 2023-03-03T06:23:59.009 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-03-03T06:23:59.010 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-03-03T06:23:59.010 INFO:tasks.ceph:[mgr] debug ms = 1 2023-03-03T06:23:59.010 INFO:tasks.ceph:[mon] debug mon = 20 2023-03-03T06:23:59.010 INFO:tasks.ceph:[mon] debug ms = 1 2023-03-03T06:23:59.010 INFO:tasks.ceph:[mon] debug paxos = 20 2023-03-03T06:23:59.011 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-03-03T06:23:59.011 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-03-03T06:23:59.011 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-03-03T06:23:59.011 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-03-03T06:23:59.011 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-03-03T06:23:59.012 INFO:tasks.ceph:[osd] debug ms = 1 2023-03-03T06:23:59.012 INFO:tasks.ceph:[osd] debug osd = 20 2023-03-03T06:23:59.012 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-03-03T06:23:59.012 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-03-03T06:23:59.012 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-03-03T06:23:59.013 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-03-03T06:23:59.013 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-03-03T06:23:59.013 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-03-03T06:23:59.013 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-03-03T06:23:59.013 INFO:tasks.ceph:Setting up mon.a... 2023-03-03T06:23:59.014 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-03-03T06:23:59.099 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.keyring 2023-03-03T06:23:59.122 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-03-03T06:23:59.219 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-03T06:23:59.263 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.3'), ('mon.b', '172.21.15.32'), ('mon.c', '[v2:172.21.15.32:3301,v1:172.21.15.32:6790]')] 2023-03-03T06:23:59.264 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.3,172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-03-03T06:23:59.264 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:23:59.265 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-03-03T06:23:59.330 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.3 --add b 172.21.15.32 --addv c '[v2:172.21.15.32:3301,v1:172.21.15.32:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:23:59.461 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:23:59.461 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: generated fsid 7d496254-7268-405f-a1d8-c8033dc8659a 2023-03-03T06:23:59.461 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = pacific 2023-03-03T06:23:59.462 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 0 2023-03-03T06:23:59.462 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 7d496254-7268-405f-a1d8-c8033dc8659a 2023-03-03T06:23:59.462 INFO:teuthology.orchestra.run.smithi003.stdout:last_changed 2023-03-03T06:23:59.459449+0000 2023-03-03T06:23:59.462 INFO:teuthology.orchestra.run.smithi003.stdout:created 2023-03-03T06:23:59.459449+0000 2023-03-03T06:23:59.462 INFO:teuthology.orchestra.run.smithi003.stdout:min_mon_release 16 (pacific) 2023-03-03T06:23:59.462 INFO:teuthology.orchestra.run.smithi003.stdout:election_strategy: 1 2023-03-03T06:23:59.462 INFO:teuthology.orchestra.run.smithi003.stdout:0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2023-03-03T06:23:59.463 INFO:teuthology.orchestra.run.smithi003.stdout:1: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.b 2023-03-03T06:23:59.463 INFO:teuthology.orchestra.run.smithi003.stdout:2: [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] mon.c 2023-03-03T06:23:59.463 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-03-03T06:23:59.468 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-03-03T06:23:59.486 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7d496254-7268-405f-a1d8-c8033dc8659a... 2023-03-03T06:23:59.488 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-03T06:23:59.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-03T06:23:59.633 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2023-03-03T06:23:59.633 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2023-03-03T06:23:59.633 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-03T06:23:59.633 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2023-03-03T06:23:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2023-03-03T06:23:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2023-03-03T06:23:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2023-03-03T06:23:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2023-03-03T06:23:59.634 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2023-03-03T06:23:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2023-03-03T06:23:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-03T06:23:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2023-03-03T06:23:59.635 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2023-03-03T06:23:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-03T06:23:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2023-03-03T06:23:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2023-03-03T06:23:59.636 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2023-03-03T06:23:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2023-03-03T06:23:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2023-03-03T06:23:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2023-03-03T06:23:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2023-03-03T06:23:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2023-03-03T06:23:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-03T06:23:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-03T06:23:59.637 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2023-03-03T06:23:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-03T06:23:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2023-03-03T06:23:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2023-03-03T06:23:59.638 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2023-03-03T06:23:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2023-03-03T06:23:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2023-03-03T06:23:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790]" 2023-03-03T06:23:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on skipped message = False 2023-03-03T06:23:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 7d496254-7268-405f-a1d8-c8033dc8659a 2023-03-03T06:23:59.639 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.640 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2023-03-03T06:23:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2023-03-03T06:23:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2023-03-03T06:23:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2023-03-03T06:23:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2023-03-03T06:23:59.640 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2023-03-03T06:23:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2023-03-03T06:23:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2023-03-03T06:23:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2023-03-03T06:23:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2023-03-03T06:23:59.641 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2023-03-03T06:23:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2023-03-03T06:23:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2023-03-03T06:23:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2023-03-03T06:23:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.642 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2023-03-03T06:23:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2023-03-03T06:23:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2023-03-03T06:23:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2023-03-03T06:23:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2023-03-03T06:23:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2023-03-03T06:23:59.643 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 20 2023-03-03T06:23:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 20 2023-03-03T06:23:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-03-03T06:23:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2023-03-03T06:23:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 10 2023-03-03T06:23:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2023-03-03T06:23:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2023-03-03T06:23:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2023-03-03T06:23:59.644 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2023-03-03T06:23:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2023-03-03T06:23:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-03-03T06:23:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.645 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2023-03-03T06:23:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-03-03T06:23:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2023-03-03T06:23:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-03-03T06:23:59.645 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-03-03T06:23:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-03-03T06:23:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-03-03T06:23:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2023-03-03T06:23:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2023-03-03T06:23:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: debug finisher = 20 2023-03-03T06:23:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.646 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2023-03-03T06:23:59.646 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-03-03T06:23:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2023-03-03T06:23:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2023-03-03T06:23:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2023-03-03T06:23:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2023-03-03T06:23:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2023-03-03T06:23:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2023-03-03T06:23:59.647 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2023-03-03T06:23:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2023-03-03T06:23:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2023-03-03T06:23:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-03T06:23:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2023-03-03T06:23:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2023-03-03T06:23:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.648 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2023-03-03T06:23:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-03T06:23:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-03T06:23:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2023-03-03T06:23:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2023-03-03T06:23:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2023-03-03T06:23:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2023-03-03T06:23:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:23:59.650 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2023-03-03T06:23:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2023-03-03T06:23:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2023-03-03T06:23:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2023-03-03T06:23:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-03T06:23:59.650 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-03T06:23:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2023-03-03T06:23:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2023-03-03T06:23:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-03-03T06:23:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2023-03-03T06:23:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-03-03T06:23:59.651 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-03-03T06:23:59.651 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2023-03-03T06:23:59.651 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2023-03-03T06:23:59.652 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2023-03-03T06:23:59.652 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-03-03T06:23:59.652 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2023-03-03T06:23:59.652 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2023-03-03T06:23:59.652 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2023-03-03T06:23:59.652 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2023-03-03T06:23:59.652 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2023-03-03T06:23:59.652 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2023-03-03T06:23:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2023-03-03T06:23:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2023-03-03T06:23:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2023-03-03T06:23:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2023-03-03T06:23:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2023-03-03T06:23:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2023-03-03T06:23:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2023-03-03T06:23:59.653 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2023-03-03T06:23:59.654 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2023-03-03T06:23:59.654 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2023-03-03T06:23:59.654 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-03T06:23:59.655 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2023-03-03T06:23:59.655 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.655 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2023-03-03T06:23:59.655 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.655 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2023-03-03T06:23:59.655 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.655 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2023-03-03T06:23:59.655 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2023-03-03T06:23:59.656 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.656 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2023-03-03T06:23:59.656 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2023-03-03T06:23:59.656 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.656 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-03T06:23:59.656 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2023-03-03T06:23:59.656 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.656 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2023-03-03T06:23:59.657 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-03T06:23:59.657 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.657 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2023-03-03T06:23:59.657 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.657 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2023-03-03T06:23:59.657 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2023-03-03T06:23:59.657 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2023-03-03T06:23:59.658 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2023-03-03T06:23:59.658 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2023-03-03T06:23:59.658 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2023-03-03T06:23:59.658 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2023-03-03T06:23:59.658 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-03T06:23:59.658 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-03T06:23:59.658 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2023-03-03T06:23:59.658 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.659 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-03T06:23:59.659 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.659 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2023-03-03T06:23:59.659 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.659 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2023-03-03T06:23:59.659 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.659 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2023-03-03T06:23:59.659 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2023-03-03T06:23:59.660 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2023-03-03T06:23:59.660 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.3,172.21.15.32,[v2:172.21.15.32:3301,v1:172.21.15.32:6790]" 2023-03-03T06:23:59.660 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on skipped message = False 2023-03-03T06:23:59.660 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = 7d496254-7268-405f-a1d8-c8033dc8659a 2023-03-03T06:23:59.660 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.660 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2023-03-03T06:23:59.660 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2023-03-03T06:23:59.660 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.661 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2023-03-03T06:23:59.661 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2023-03-03T06:23:59.661 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.661 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2023-03-03T06:23:59.661 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2023-03-03T06:23:59.661 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.661 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2023-03-03T06:23:59.662 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2023-03-03T06:23:59.662 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2023-03-03T06:23:59.662 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.662 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2023-03-03T06:23:59.662 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2023-03-03T06:23:59.662 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.662 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2023-03-03T06:23:59.663 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.663 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2023-03-03T06:23:59.663 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.663 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2023-03-03T06:23:59.663 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.663 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2023-03-03T06:23:59.663 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2023-03-03T06:23:59.663 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.664 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2023-03-03T06:23:59.664 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2023-03-03T06:23:59.664 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2023-03-03T06:23:59.664 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2023-03-03T06:23:59.664 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 20 2023-03-03T06:23:59.664 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 20 2023-03-03T06:23:59.664 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-03-03T06:23:59.664 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2023-03-03T06:23:59.665 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 10 2023-03-03T06:23:59.665 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2023-03-03T06:23:59.665 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2023-03-03T06:23:59.665 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2023-03-03T06:23:59.665 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2023-03-03T06:23:59.665 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2023-03-03T06:23:59.665 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2023-03-03T06:23:59.666 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.666 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2023-03-03T06:23:59.666 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-03-03T06:23:59.666 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2023-03-03T06:23:59.666 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2023-03-03T06:23:59.666 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2023-03-03T06:23:59.666 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2023-03-03T06:23:59.667 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2023-03-03T06:23:59.667 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2023-03-03T06:23:59.667 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2023-03-03T06:23:59.667 INFO:teuthology.orchestra.run.smithi032.stdout: debug finisher = 20 2023-03-03T06:23:59.667 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.667 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2023-03-03T06:23:59.667 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-03-03T06:23:59.667 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2023-03-03T06:23:59.668 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2023-03-03T06:23:59.668 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2023-03-03T06:23:59.668 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2023-03-03T06:23:59.668 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2023-03-03T06:23:59.668 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2023-03-03T06:23:59.668 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2023-03-03T06:23:59.668 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2023-03-03T06:23:59.669 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2023-03-03T06:23:59.669 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.669 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-03T06:23:59.669 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2023-03-03T06:23:59.669 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2023-03-03T06:23:59.670 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.670 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2023-03-03T06:23:59.670 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-03T06:23:59.670 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-03T06:23:59.670 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.670 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2023-03-03T06:23:59.671 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2023-03-03T06:23:59.671 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.671 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2023-03-03T06:23:59.671 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2023-03-03T06:23:59.671 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-03-03T06:23:59.671 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2023-03-03T06:23:59.671 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2023-03-03T06:23:59.671 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2023-03-03T06:23:59.672 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2023-03-03T06:23:59.672 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-03T06:23:59.672 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-03T06:23:59.672 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2023-03-03T06:23:59.672 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2023-03-03T06:23:59.672 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-03-03T06:23:59.672 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = "" 2023-03-03T06:23:59.672 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2023-03-03T06:23:59.673 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2023-03-03T06:23:59.673 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2023-03-03T06:23:59.673 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2023-03-03T06:23:59.673 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2023-03-03T06:23:59.673 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2023-03-03T06:23:59.673 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2023-03-03T06:23:59.673 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2023-03-03T06:23:59.674 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2023-03-03T06:23:59.674 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2023-03-03T06:23:59.674 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2023-03-03T06:23:59.674 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds balancer = 20 2023-03-03T06:23:59.674 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-03-03T06:23:59.674 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2023-03-03T06:23:59.674 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2023-03-03T06:23:59.674 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2023-03-03T06:23:59.675 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2023-03-03T06:23:59.675 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2023-03-03T06:23:59.675 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2023-03-03T06:23:59.675 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2023-03-03T06:23:59.678 INFO:tasks.ceph:Creating admin key on mon.a... 2023-03-03T06:23:59.678 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-03-03T06:23:59.801 INFO:tasks.ceph:Copying monmap to all nodes... 2023-03-03T06:23:59.802 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:23:59.802 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-03-03T06:23:59.822 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:23:59.822 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-03-03T06:23:59.887 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi003.front.sepia.ceph.com 2023-03-03T06:23:59.888 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:23:59.889 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-03T06:23:59.889 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-03T06:24:00.008 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:00.008 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:24:00.075 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi032.front.sepia.ceph.com 2023-03-03T06:24:00.075 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:00.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-03T06:24:00.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-03T06:24:00.157 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:00.158 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:24:00.220 INFO:tasks.ceph:Setting up mon nodes... 2023-03-03T06:24:00.221 INFO:tasks.ceph:Setting up mgr nodes... 2023-03-03T06:24:00.221 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-03-03T06:24:00.354 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-03-03T06:24:00.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-03-03T06:24:00.494 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-03-03T06:24:00.518 INFO:tasks.ceph:Setting up mds nodes... 2023-03-03T06:24:00.518 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-03-03T06:24:00.655 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-03-03T06:24:00.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-03-03T06:24:00.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-03-03T06:24:00.880 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-03-03T06:24:00.895 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-03-03T06:24:00.940 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-03-03T06:24:01.072 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-03-03T06:24:01.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-03-03T06:24:01.132 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-03-03T06:24:01.297 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-03-03T06:24:01.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-03-03T06:24:01.353 INFO:tasks.ceph_client:Setting up client nodes... 2023-03-03T06:24:01.354 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-03-03T06:24:01.443 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-03-03T06:24:01.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-03-03T06:24:01.574 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-03-03T06:24:01.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-03-03T06:24:01.712 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-03-03T06:24:01.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-03-03T06:24:01.848 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-03-03T06:24:01.890 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-03-03T06:24:01.891 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-03-03T06:24:01.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-03-03T06:24:01.938 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-03T06:24:01.939 INFO:tasks.ceph:role: osd.0 2023-03-03T06:24:01.939 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com 2023-03-03T06:24:01.939 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-03T06:24:02.546 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-03T06:24:02.546 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-03T06:24:02.547 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-03T06:24:02.547 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 2023-03-03T06:24:02.547 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-03T06:24:02.547 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-03-03T06:24:02.547 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-03T06:24:02.547 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-03T06:24:02.547 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-03T06:24:02.548 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-03T06:24:02.552 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-03-03T06:24:02.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-03-03T06:24:02.687 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-03-03T06:24:02.714 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2023-03-03T06:24:02.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-03T06:24:02.716 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-03-03T06:24:02.802 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-03T06:24:02.803 INFO:tasks.ceph:role: osd.1 2023-03-03T06:24:02.803 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com 2023-03-03T06:24:02.803 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-03T06:24:03.337 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-03T06:24:03.338 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-03T06:24:03.338 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-03T06:24:03.338 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 2023-03-03T06:24:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-03T06:24:03.339 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-03-03T06:24:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-03T06:24:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-03T06:24:03.339 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-03T06:24:03.339 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-03T06:24:03.347 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-03-03T06:24:03.347 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-03-03T06:24:03.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-03-03T06:24:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2023-03-03T06:24:03.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-03T06:24:03.467 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-03-03T06:24:03.556 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-03T06:24:03.557 INFO:tasks.ceph:role: osd.2 2023-03-03T06:24:03.557 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com 2023-03-03T06:24:03.557 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-03-03T06:24:04.184 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-03-03T06:24:04.184 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-03T06:24:04.184 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-03T06:24:04.185 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 2023-03-03T06:24:04.185 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-03T06:24:04.185 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-03-03T06:24:04.185 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-03T06:24:04.185 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-03T06:24:04.185 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-03T06:24:04.186 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-03T06:24:04.198 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-03-03T06:24:04.198 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-03-03T06:24:04.352 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-03-03T06:24:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2023-03-03T06:24:04.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-03T06:24:04.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-03-03T06:24:04.467 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-03T06:24:04.468 INFO:tasks.ceph:role: osd.3 2023-03-03T06:24:04.468 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com 2023-03-03T06:24:04.468 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-03-03T06:24:05.047 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-03-03T06:24:05.047 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-03T06:24:05.047 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-03T06:24:05.047 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 2023-03-03T06:24:05.047 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-03T06:24:05.048 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2023-03-03T06:24:05.048 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-03T06:24:05.048 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-03T06:24:05.048 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-03T06:24:05.048 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-03T06:24:05.055 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2023-03-03T06:24:05.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-03-03T06:24:05.121 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-03-03T06:24:05.146 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2023-03-03T06:24:05.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-03T06:24:05.148 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:24:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:05.324+0000 7fbc71b3a700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-03-03T06:24:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:05.324+0000 7fbc71b3a700 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-03-03T06:24:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:05.324+0000 7fbc71b3a700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-03T06:24:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:05.324+0000 7fbc71b3a700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-03T06:24:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:05.324+0000 7fbc71b3a700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-03T06:24:05.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:05.328+0000 7fbc71b3a700 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-03-03T06:24:08.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-03-03T06:24:08.767 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:24:08.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:08.908+0000 7fb4c1445700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-03-03T06:24:08.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:08.908+0000 7fb4c1445700 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-03-03T06:24:08.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:08.908+0000 7fb4c1445700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-03T06:24:08.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:08.908+0000 7fb4c1445700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-03T06:24:08.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:08.908+0000 7fb4c1445700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-03T06:24:08.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:08.912+0000 7fb4c1445700 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-03-03T06:24:12.287 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-03-03T06:24:12.334 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:24:12.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:12.480+0000 7fa69dd82700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-03-03T06:24:12.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:12.484+0000 7fa69dd82700 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-03-03T06:24:12.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:12.484+0000 7fa69dd82700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-03T06:24:12.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:12.484+0000 7fa69dd82700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-03T06:24:12.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:12.484+0000 7fa69dd82700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-03T06:24:12.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:12.484+0000 7fa69dd82700 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-03-03T06:24:15.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-03-03T06:24:15.905 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:24:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:16.056+0000 7f2a79bea700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-03-03T06:24:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:16.056+0000 7f2a79bea700 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-03-03T06:24:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:16.056+0000 7f2a79bea700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-03T06:24:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:16.056+0000 7f2a79bea700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-03T06:24:16.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:16.056+0000 7f2a79bea700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-03T06:24:16.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:16.060+0000 7f2a79bea700 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-03-03T06:24:19.459 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-03-03T06:24:19.501 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-03-03T06:24:19.548 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-03T06:24:19.548 INFO:tasks.ceph:role: osd.4 2023-03-03T06:24:19.549 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com 2023-03-03T06:24:19.549 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-03T06:24:19.675 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-03T06:24:19.675 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-03T06:24:19.675 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-03T06:24:19.676 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 2023-03-03T06:24:19.676 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-03T06:24:19.676 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-03-03T06:24:19.676 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-03T06:24:19.676 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-03T06:24:19.676 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-03T06:24:19.677 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-03T06:24:19.681 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-03-03T06:24:19.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-03-03T06:24:19.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-03-03T06:24:19.811 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2023-03-03T06:24:19.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-03T06:24:19.813 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-03-03T06:24:19.902 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-03T06:24:19.903 INFO:tasks.ceph:role: osd.5 2023-03-03T06:24:19.903 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com 2023-03-03T06:24:19.904 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-03T06:24:20.026 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-03T06:24:20.027 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-03T06:24:20.027 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-03T06:24:20.027 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 2023-03-03T06:24:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-03T06:24:20.027 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-03-03T06:24:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-03T06:24:20.027 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-03T06:24:20.028 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-03T06:24:20.028 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-03T06:24:20.036 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-03-03T06:24:20.037 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-03-03T06:24:20.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-03-03T06:24:20.110 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2023-03-03T06:24:20.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-03T06:24:20.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-03-03T06:24:20.197 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-03T06:24:20.198 INFO:tasks.ceph:role: osd.6 2023-03-03T06:24:20.198 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com 2023-03-03T06:24:20.198 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-03-03T06:24:20.322 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-03-03T06:24:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-03T06:24:20.322 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-03T06:24:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 2023-03-03T06:24:20.323 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-03T06:24:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-03-03T06:24:20.323 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-03T06:24:20.323 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-03T06:24:20.323 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-03T06:24:20.323 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-03T06:24:20.330 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-03-03T06:24:20.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-03-03T06:24:20.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-03-03T06:24:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2023-03-03T06:24:20.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-03T06:24:20.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-03-03T06:24:20.538 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-03T06:24:20.539 INFO:tasks.ceph:role: osd.7 2023-03-03T06:24:20.539 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com 2023-03-03T06:24:20.539 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-03-03T06:24:20.657 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-03-03T06:24:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-03T06:24:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-03T06:24:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 2023-03-03T06:24:20.658 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-03T06:24:20.658 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-03-03T06:24:20.658 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-03T06:24:20.658 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-03T06:24:20.659 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-03T06:24:20.659 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-03T06:24:20.665 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-03-03T06:24:20.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-03-03T06:24:20.716 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-03-03T06:24:20.743 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2023-03-03T06:24:20.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-03T06:24:20.745 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:24:20.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:20.927+0000 7f0fb8e95700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-03-03T06:24:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:20.927+0000 7f0fb8e95700 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-03-03T06:24:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:20.927+0000 7f0fb8e95700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-03T06:24:20.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:20.927+0000 7f0fb8e95700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-03T06:24:20.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:20.927+0000 7f0fb8e95700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-03T06:24:20.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:20.931+0000 7f0fb8e95700 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-03-03T06:24:24.288 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-03-03T06:24:24.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:24:24.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:24.475+0000 7fc92cac3700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-03-03T06:24:24.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:24.479+0000 7fc92cac3700 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-03-03T06:24:24.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:24.479+0000 7fc92cac3700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-03T06:24:24.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:24.479+0000 7fc92cac3700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-03T06:24:24.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:24.479+0000 7fc92cac3700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-03T06:24:24.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:24.479+0000 7fc92cac3700 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-03-03T06:24:26.964 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:24:26.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:24:27.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-03-03T06:24:27.886 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:24:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:28.031+0000 7fd4c91d5700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-03-03T06:24:28.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:28.031+0000 7fd4c91d5700 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-03-03T06:24:28.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:28.035+0000 7fd4c91d5700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-03T06:24:28.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:28.035+0000 7fd4c91d5700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-03T06:24:28.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:28.035+0000 7fd4c91d5700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-03T06:24:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:28.035+0000 7fd4c91d5700 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-03-03T06:24:31.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-03-03T06:24:31.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:24:31.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:31.611+0000 7f8950a72700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-03-03T06:24:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:31.611+0000 7f8950a72700 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-03-03T06:24:31.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:31.611+0000 7f8950a72700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-03T06:24:31.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:31.611+0000 7f8950a72700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-03T06:24:31.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:31.611+0000 7f8950a72700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-03T06:24:31.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:31.615+0000 7f8950a72700 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-03-03T06:24:34.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-03-03T06:24:35.035 INFO:tasks.ceph:Reading keys from all nodes... 2023-03-03T06:24:35.035 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:35.036 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-03-03T06:24:35.081 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:35.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-03-03T06:24:35.167 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:35.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-03-03T06:24:35.256 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:35.256 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-03-03T06:24:35.348 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:35.349 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-03-03T06:24:35.430 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:35.431 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-03-03T06:24:35.520 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:35.520 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-03-03T06:24:35.604 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:35.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-03-03T06:24:35.649 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:35.650 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-03-03T06:24:35.737 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:35.737 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-03-03T06:24:35.825 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:35.825 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-03-03T06:24:35.914 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:35.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-03-03T06:24:36.002 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:36.003 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-03-03T06:24:36.087 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:36.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-03-03T06:24:36.178 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:36.179 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-03-03T06:24:36.200 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:36.201 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-03-03T06:24:36.266 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:36.267 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-03-03T06:24:36.288 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:36.289 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-03-03T06:24:36.352 INFO:tasks.ceph:Adding keys to all mons... 2023-03-03T06:24:36.353 DEBUG:teuthology.orchestra.run.smithi003:> sudo tee -a /etc/ceph/ceph.keyring 2023-03-03T06:24:36.361 DEBUG:teuthology.orchestra.run.smithi032:> sudo tee -a /etc/ceph/ceph.keyring 2023-03-03T06:24:36.406 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.y] 2023-03-03T06:24:36.406 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCAkgFkjLAIFRAA5hYq+AKWH11R3giPKunqog== 2023-03-03T06:24:36.406 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2023-03-03T06:24:36.406 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCAkgFkfHvwJhAAPEj36Q5cJyrlQYjEaG2sGg== 2023-03-03T06:24:36.406 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2023-03-03T06:24:36.407 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCAkgFkGgRnNBAA5+REl/roeIhnxFvAmB5okw== 2023-03-03T06:24:36.407 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.0] 2023-03-03T06:24:36.407 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCFkgFkuEucExAA3ukOiXqZpzCZqHrdMkpr7w== 2023-03-03T06:24:36.407 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.1] 2023-03-03T06:24:36.407 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCIkgFkAWd2NhAA3X8Z7xuVlRU2lYChQJVWKg== 2023-03-03T06:24:36.407 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.2] 2023-03-03T06:24:36.408 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCMkgFkbIQTHRAAyyfQ1Qc3LkCwNrOYWhM3DA== 2023-03-03T06:24:36.408 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.3] 2023-03-03T06:24:36.408 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCQkgFkQhK8AxAAm58cosvi5WKSGo+ez131gg== 2023-03-03T06:24:36.408 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.x] 2023-03-03T06:24:36.408 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCAkgFkNkJqHRAACPVn302t25/DzryFbFr12w== 2023-03-03T06:24:36.408 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2023-03-03T06:24:36.408 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCBkgFkTOo9BBAA78PiqKJ59InZKdWZFMCN7w== 2023-03-03T06:24:36.408 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2023-03-03T06:24:36.409 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCBkgFkRmqnERAAd8kObftCMq5QZDGFzcbtWw== 2023-03-03T06:24:36.409 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.4] 2023-03-03T06:24:36.409 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCUkgFkmRKDNxAABkz6ksqRui2bySWPQQfBEg== 2023-03-03T06:24:36.409 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.5] 2023-03-03T06:24:36.409 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCYkgFkIhe8HBAAyvuIJoh5EB4nGj6S443d1A== 2023-03-03T06:24:36.409 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.6] 2023-03-03T06:24:36.410 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCckgFkEOgvAhAAcQBWJ6xLJ4JY8KDnosIRGQ== 2023-03-03T06:24:36.410 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.7] 2023-03-03T06:24:36.410 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCfkgFk+yavJBAAEDFnVa1ZIFMR8X9asFniNA== 2023-03-03T06:24:36.410 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2023-03-03T06:24:36.410 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCBkgFkq25UGhAAPQ0Adr+W73H75r08iuEYDQ== 2023-03-03T06:24:36.410 INFO:teuthology.orchestra.run.smithi003.stdout:[client.2] 2023-03-03T06:24:36.410 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCBkgFkgHYgIhAAKg+r8B3OOYp8KU9Oly+U1Q== 2023-03-03T06:24:36.410 INFO:teuthology.orchestra.run.smithi003.stdout:[client.1] 2023-03-03T06:24:36.411 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCBkgFkPYhfKhAAPtcpq/S9ReAFseRm31sPzg== 2023-03-03T06:24:36.411 INFO:teuthology.orchestra.run.smithi003.stdout:[client.3] 2023-03-03T06:24:36.411 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCBkgFkPL17MhAAKe5MQgwKkfsfuhe4Q49STg== 2023-03-03T06:24:36.436 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.y] 2023-03-03T06:24:36.437 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCAkgFkjLAIFRAA5hYq+AKWH11R3giPKunqog== 2023-03-03T06:24:36.437 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2023-03-03T06:24:36.437 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCAkgFkfHvwJhAAPEj36Q5cJyrlQYjEaG2sGg== 2023-03-03T06:24:36.437 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2023-03-03T06:24:36.437 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCAkgFkGgRnNBAA5+REl/roeIhnxFvAmB5okw== 2023-03-03T06:24:36.437 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.0] 2023-03-03T06:24:36.438 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCFkgFkuEucExAA3ukOiXqZpzCZqHrdMkpr7w== 2023-03-03T06:24:36.438 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.1] 2023-03-03T06:24:36.438 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCIkgFkAWd2NhAA3X8Z7xuVlRU2lYChQJVWKg== 2023-03-03T06:24:36.438 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.2] 2023-03-03T06:24:36.438 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCMkgFkbIQTHRAAyyfQ1Qc3LkCwNrOYWhM3DA== 2023-03-03T06:24:36.438 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.3] 2023-03-03T06:24:36.438 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCQkgFkQhK8AxAAm58cosvi5WKSGo+ez131gg== 2023-03-03T06:24:36.439 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.x] 2023-03-03T06:24:36.439 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCAkgFkNkJqHRAACPVn302t25/DzryFbFr12w== 2023-03-03T06:24:36.439 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2023-03-03T06:24:36.439 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCBkgFkTOo9BBAA78PiqKJ59InZKdWZFMCN7w== 2023-03-03T06:24:36.439 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2023-03-03T06:24:36.439 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCBkgFkRmqnERAAd8kObftCMq5QZDGFzcbtWw== 2023-03-03T06:24:36.439 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.4] 2023-03-03T06:24:36.439 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCUkgFkmRKDNxAABkz6ksqRui2bySWPQQfBEg== 2023-03-03T06:24:36.440 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.5] 2023-03-03T06:24:36.440 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCYkgFkIhe8HBAAyvuIJoh5EB4nGj6S443d1A== 2023-03-03T06:24:36.440 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.6] 2023-03-03T06:24:36.440 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCckgFkEOgvAhAAcQBWJ6xLJ4JY8KDnosIRGQ== 2023-03-03T06:24:36.440 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.7] 2023-03-03T06:24:36.440 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCfkgFk+yavJBAAEDFnVa1ZIFMR8X9asFniNA== 2023-03-03T06:24:36.440 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-03-03T06:24:36.440 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCBkgFkq25UGhAAPQ0Adr+W73H75r08iuEYDQ== 2023-03-03T06:24:36.441 INFO:teuthology.orchestra.run.smithi032.stdout:[client.2] 2023-03-03T06:24:36.441 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCBkgFkgHYgIhAAKg+r8B3OOYp8KU9Oly+U1Q== 2023-03-03T06:24:36.441 INFO:teuthology.orchestra.run.smithi032.stdout:[client.1] 2023-03-03T06:24:36.441 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCBkgFkPYhfKhAAPtcpq/S9ReAFseRm31sPzg== 2023-03-03T06:24:36.441 INFO:teuthology.orchestra.run.smithi032.stdout:[client.3] 2023-03-03T06:24:36.441 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQCBkgFkPL17MhAAKe5MQgwKkfsfuhe4Q49STg== 2023-03-03T06:24:36.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-03T06:24:36.460 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-03T06:24:36.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-03T06:24:36.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-03T06:24:36.703 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-03T06:24:36.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-03T06:24:36.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:36.825 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:36.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:36.936 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:37.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:37.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:37.152 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:37.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:37.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-03T06:24:37.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-03T06:24:37.382 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-03T06:24:37.391 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-03T06:24:37.496 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-03T06:24:37.503 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-03T06:24:37.613 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:37.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:37.726 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:37.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:37.838 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:37.845 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:37.948 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:37.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-03T06:24:38.060 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-03T06:24:38.067 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-03T06:24:38.180 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-03T06:24:38.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-03T06:24:38.293 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-03T06:24:38.301 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-03T06:24:38.409 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-03T06:24:38.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-03T06:24:38.522 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-03-03T06:24:38.522 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-03-03T06:24:38.566 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-03T06:24:38.748 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-03-03T06:24:38.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-03-03T06:24:38.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-03T06:24:39.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-03-03T06:24:39.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-03-03T06:24:39.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-03T06:24:39.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-03-03T06:24:39.381 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:24:39.390 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-03-03T06:24:39.448 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-03-03T06:24:39.448 INFO:tasks.ceph.mon.a:Restarting daemon 2023-03-03T06:24:39.449 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-03-03T06:24:39.456 INFO:tasks.ceph.mon.a:Started 2023-03-03T06:24:39.457 INFO:tasks.ceph.mon.b:Restarting daemon 2023-03-03T06:24:39.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-03-03T06:24:39.502 INFO:tasks.ceph.mon.b:Started 2023-03-03T06:24:39.503 INFO:tasks.ceph.mon.c:Restarting daemon 2023-03-03T06:24:39.503 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-03-03T06:24:39.508 INFO:tasks.ceph.mon.c:Started 2023-03-03T06:24:39.508 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-03-03T06:24:39.508 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-03-03T06:24:39.509 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-03-03T06:24:39.517 INFO:tasks.ceph.mgr.y:Started 2023-03-03T06:24:39.517 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-03-03T06:24:39.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-03-03T06:24:39.522 INFO:tasks.ceph.mgr.x:Started 2023-03-03T06:24:39.523 INFO:tasks.ceph:Setting crush tunables to default 2023-03-03T06:24:39.523 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd crush tunables default 2023-03-03T06:24:39.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.704+0000 7fbebaf77700 1 Processor -- start 2023-03-03T06:24:39.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.704+0000 7fbebaf77700 1 -- start start 2023-03-03T06:24:39.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.704+0000 7fbebaf77700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:39.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.704+0000 7fbebaf77700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:39.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.704+0000 7fbebaf77700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb415d940 con 0x7fbeb415c370 2023-03-03T06:24:39.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.704+0000 7fbebaf77700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb40ce520 con 0x7fbeb40cb520 2023-03-03T06:24:39.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.708+0000 7fbebaf77700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb40ccbc0 con 0x7fbeb40b0ed0 2023-03-03T06:24:39.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.708+0000 7fbeba776700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:39.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.708+0000 7fbeb9774700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:39.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.708+0000 7fbeba776700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60822/0 (socket says 172.21.15.3:60822) 2023-03-03T06:24:39.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.708+0000 7fbeba776700 1 -- 172.21.15.3:0/3755031991 learned_addr learned my addr 172.21.15.3:0/3755031991 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:24:39.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.708+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 msgr2=0x7fbeb406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-03-03T06:24:39.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.708+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 msgr2=0x7fbeb406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:39.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.708+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:39.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.708+0000 7fbeba776700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 msgr2=0x7fbeb415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-03-03T06:24:39.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.708+0000 7fbeba776700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 msgr2=0x7fbeb415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:39.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.708+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:39.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.708+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:39.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.708+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:39.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.912+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:39.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.912+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:39.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.912+0000 7fbeba776700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 msgr2=0x7fbeb415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-03-03T06:24:39.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.912+0000 7fbeba776700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 msgr2=0x7fbeb415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:39.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.912+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:39.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.912+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 msgr2=0x7fbeb406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-03-03T06:24:39.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.912+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 msgr2=0x7fbeb406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:39.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.912+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:39.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.912+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-03-03T06:24:39.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:39.912+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-03-03T06:24:40.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:40.316+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:40.316+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:40.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:40.316+0000 7fbeba776700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 msgr2=0x7fbeb415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-03-03T06:24:40.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:40.316+0000 7fbeba776700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 msgr2=0x7fbeb415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:40.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:40.316+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:40.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:40.316+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-03-03T06:24:40.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:40.316+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 msgr2=0x7fbeb406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-03-03T06:24:40.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:40.316+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 msgr2=0x7fbeb406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:40.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:40.316+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:40.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:40.316+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-03-03T06:24:41.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:41.120+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:41.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:41.120+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:41.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:41.120+0000 7fbeba776700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 msgr2=0x7fbeb415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-03-03T06:24:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:41.120+0000 7fbeba776700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 msgr2=0x7fbeb415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:41.120+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:41.120+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 msgr2=0x7fbeb406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-03-03T06:24:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:41.120+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 msgr2=0x7fbeb406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:41.120+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:41.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:41.120+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-03-03T06:24:41.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:41.120+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-03-03T06:24:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.708+0000 7fbeab7fe700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 msgr2=0x7fbeb406c090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-03T06:24:42.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.708+0000 7fbeab7fe700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.32:3300/0 conn(0x7fbeb40b0ed0 0x7fbeb406c090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.708+0000 7fbeab7fe700 1 -- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 msgr2=0x7fbeb415d400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-03T06:24:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.708+0000 7fbeab7fe700 1 --2- 172.21.15.3:0/3755031991 >> v2:172.21.15.3:3300/0 conn(0x7fbeb40cb520 0x7fbeb415d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.708+0000 7fbeab7fe700 1 -- 172.21.15.3:0/3755031991 >> v1:172.21.15.3:6789/0 conn(0x7fbeb415c370 legacy=0x7fbeb40b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.708+0000 7fbeab7fe700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.708+0000 7fbeab7fe700 1 -- 172.21.15.3:0/3755031991 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb40cbae0 con 0x7fbe98003730 2023-03-03T06:24:42.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.708+0000 7fbeab7fe700 1 -- 172.21.15.3:0/3755031991 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40ccbc0 con 0x7fbe98007370 2023-03-03T06:24:42.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.708+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:42.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.708+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 msgr2=0x7fbe980097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-03-03T06:24:42.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.708+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 msgr2=0x7fbe980097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:42.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.708+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:42.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.708+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.912+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.912+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 msgr2=0x7fbe980097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-03-03T06:24:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.912+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 msgr2=0x7fbe980097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.912+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:42.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:42.912+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-03-03T06:24:43.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:43.316+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:43.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:43.316+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 msgr2=0x7fbe980097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-03-03T06:24:43.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:43.316+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 msgr2=0x7fbe980097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:43.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:43.316+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:43.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:43.316+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-03-03T06:24:44.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.120+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:44.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.120+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 msgr2=0x7fbe980097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-03-03T06:24:44.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.120+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 msgr2=0x7fbe980097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.120+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:44.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.120+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-03-03T06:24:44.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.884+0000 7fbeb8f73700 1 -- 172.21.15.3:0/3755031991 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402116935 0 0) 0x7fbeb40cbae0 con 0x7fbe98003730 2023-03-03T06:24:44.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.884+0000 7fbeb8f73700 1 -- 172.21.15.3:0/3755031991 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbea00035d0 con 0x7fbe98003730 2023-03-03T06:24:44.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.888+0000 7fbeb8f73700 1 -- 172.21.15.3:0/3755031991 <== mon.1 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 480389755 0 0) 0x7fbea00035d0 con 0x7fbe98003730 2023-03-03T06:24:44.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.888+0000 7fbeb8f73700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 msgr2=0x7fbe980097e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-03T06:24:44.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.888+0000 7fbeb8f73700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbe980097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:44.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.888+0000 7fbeb8f73700 1 -- 172.21.15.3:0/3755031991 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9800afa0 con 0x7fbe98003730 2023-03-03T06:24:44.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.888+0000 7fbeb8f73700 1 -- 172.21.15.3:0/3755031991 <== mon.1 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7fbeb00016f0 con 0x7fbe98003730 2023-03-03T06:24:44.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.888+0000 7fbeb8f73700 1 -- 172.21.15.3:0/3755031991 >> v1:172.21.15.32:6789/0 conn(0x7fbe98003730 legacy=0x7fbe98005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:44.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.888+0000 7fbeb8f73700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbea0003a20 0x7fbea0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:44.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.888+0000 7fbeb8f73700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea0004870 0x7fbea0004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:44.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.888+0000 7fbeb8f73700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbea000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:44.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.888+0000 7fbeb8f73700 1 -- 172.21.15.3:0/3755031991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40ccbc0 con 0x7fbea0003a20 2023-03-03T06:24:44.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.888+0000 7fbeb8f73700 1 -- 172.21.15.3:0/3755031991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea0005580 con 0x7fbea0004870 2023-03-03T06:24:44.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.888+0000 7fbeb8f73700 1 -- 172.21.15.3:0/3755031991 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea0005740 con 0x7fbe98007370 2023-03-03T06:24:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeba776700 1 -- 172.21.15.3:0/3755031991 reap_dead start 2023-03-03T06:24:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea0004870 0x7fbea0004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeb9f75700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbea0003a20 0x7fbea0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbea000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:44.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 msgr2=0x7fbea000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeba776700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 msgr2=0x7fbea000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-03-03T06:24:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeba776700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 msgr2=0x7fbea000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbea000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeba776700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbea000ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbea000ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbea0003a20 msgr2=0x7fbea0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:44.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbea0003a20 0x7fbea0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:44.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeb9774700 1 -- 172.21.15.3:0/3755031991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea00035d0 con 0x7fbea0004870 2023-03-03T06:24:44.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.892+0000 7fbeb9774700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea0004870 0x7fbea0004ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbea4019b20 tx=0x7fbea00058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d42c485bfdd768b8 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:44.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.896+0000 7fbeb8f73700 1 -- 172.21.15.3:0/3755031991 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea401e070 con 0x7fbea0004870 2023-03-03T06:24:44.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.896+0000 7fbeb8f73700 1 -- 172.21.15.3:0/3755031991 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea400c040 con 0x7fbea0004870 2023-03-03T06:24:44.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.896+0000 7fbeb8f73700 1 -- 172.21.15.3:0/3755031991 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4016b00 con 0x7fbea0004870 2023-03-03T06:24:44.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.896+0000 7fbebaf77700 1 -- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea0004870 msgr2=0x7fbea0004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.896+0000 7fbebaf77700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea0004870 0x7fbea0004ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbea4019b20 tx=0x7fbea00058a0 comp rx=0 tx=0).stop 2023-03-03T06:24:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.896+0000 7fbebaf77700 1 -- 172.21.15.3:0/3755031991 shutdown_connections 2023-03-03T06:24:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.896+0000 7fbebaf77700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe98007370 0x7fbea000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:44.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.896+0000 7fbebaf77700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea0004870 0x7fbea0004ce0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:44.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.896+0000 7fbebaf77700 1 --2- 172.21.15.3:0/3755031991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbea0003a20 0x7fbea0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:44.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.896+0000 7fbebaf77700 1 -- 172.21.15.3:0/3755031991 >> 172.21.15.3:0/3755031991 conn(0x7fbeb40607c0 msgr2=0x7fbeb4060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:44.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.900+0000 7fbebaf77700 1 -- 172.21.15.3:0/3755031991 shutdown_connections 2023-03-03T06:24:44.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.900+0000 7fbebaf77700 1 -- 172.21.15.3:0/3755031991 wait complete. 2023-03-03T06:24:44.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.904+0000 7fbebaf77700 1 Processor -- start 2023-03-03T06:24:44.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.904+0000 7fbebaf77700 1 -- start start 2023-03-03T06:24:44.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.904+0000 7fbebaf77700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe98007370 0x7fbeb414bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:44.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.904+0000 7fbebaf77700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea0003a20 0x7fbeb41e6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:44.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.904+0000 7fbebaf77700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbea0004870 0x7fbeb41e6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:44.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.904+0000 7fbebaf77700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb414c520 con 0x7fbe98007370 2023-03-03T06:24:44.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.904+0000 7fbebaf77700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb414c800 con 0x7fbea0003a20 2023-03-03T06:24:44.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.904+0000 7fbebaf77700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb414cae0 con 0x7fbea0004870 2023-03-03T06:24:44.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeb9f75700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe98007370 0x7fbeb414bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:44.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeb9f75700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe98007370 0x7fbeb414bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42706/0 (socket says 172.21.15.3:42706) 2023-03-03T06:24:44.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeb9f75700 1 -- 172.21.15.3:0/2411570141 learned_addr learned my addr 172.21.15.3:0/2411570141 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:24:44.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeb9774700 1 --2- 172.21.15.3:0/2411570141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea0003a20 0x7fbeb41e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:44.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeba776700 1 --2- 172.21.15.3:0/2411570141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbea0004870 0x7fbeb41e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeba776700 1 -- 172.21.15.3:0/2411570141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbea0004870 msgr2=0x7fbeb41e6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-03-03T06:24:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeba776700 1 -- 172.21.15.3:0/2411570141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbea0004870 msgr2=0x7fbeb41e6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-03-03T06:24:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeba776700 1 --2- 172.21.15.3:0/2411570141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbea0004870 0x7fbeb41e6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeba776700 1 --2- 172.21.15.3:0/2411570141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbea0004870 0x7fbeb41e6580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeb9f75700 1 -- 172.21.15.3:0/2411570141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbea0004870 msgr2=0x7fbeb41e6580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:24:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeb9f75700 1 --2- 172.21.15.3:0/2411570141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbea0004870 0x7fbeb41e6580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:44.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeb9f75700 1 -- 172.21.15.3:0/2411570141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea0003a20 msgr2=0x7fbeb41e6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:44.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeb9f75700 1 --2- 172.21.15.3:0/2411570141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea0003a20 0x7fbeb41e6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:44.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeb9f75700 1 -- 172.21.15.3:0/2411570141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb40cc7e0 con 0x7fbe98007370 2023-03-03T06:24:44.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeb9f75700 1 --2- 172.21.15.3:0/2411570141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe98007370 0x7fbeb414bcd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0007fd0 tx=0x7fbeb000fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:44.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeaa7fc700 1 -- 172.21.15.3:0/2411570141 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb00050d0 con 0x7fbe98007370 2023-03-03T06:24:44.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeaa7fc700 1 -- 172.21.15.3:0/2411570141 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb0005250 con 0x7fbe98007370 2023-03-03T06:24:44.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeaa7fc700 1 -- 172.21.15.3:0/2411570141 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb00036a0 con 0x7fbe98007370 2023-03-03T06:24:44.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbebaf77700 1 -- 172.21.15.3:0/2411570141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb40cc500 con 0x7fbe98007370 2023-03-03T06:24:44.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbebaf77700 1 -- 172.21.15.3:0/2411570141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb415c740 con 0x7fbe98007370 2023-03-03T06:24:44.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeaa7fc700 1 -- 172.21.15.3:0/2411570141 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fbeb001b030 con 0x7fbe98007370 2023-03-03T06:24:44.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbeaa7fc700 1 -- 172.21.15.3:0/2411570141 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbeb0014c20 con 0x7fbe98007370 2023-03-03T06:24:44.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.908+0000 7fbebaf77700 1 -- 172.21.15.3:0/2411570141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb4073e90 con 0x7fbe98007370 2023-03-03T06:24:44.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:44.916+0000 7fbeaa7fc700 1 -- 172.21.15.3:0/2411570141 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fbeb0019070 con 0x7fbe98007370 2023-03-03T06:24:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:45.000+0000 7fbebaf77700 1 -- 172.21.15.3:0/2411570141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbeb40ae700 con 0x7fbe98007370 2023-03-03T06:24:45.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:45.008+0000 7fbeaa7fc700 1 -- 172.21.15.3:0/2411570141 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbeb415d940 con 0x7fbe98007370 2023-03-03T06:24:45.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:45.008+0000 7fbebaf77700 1 -- 172.21.15.3:0/2411570141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe98007370 msgr2=0x7fbeb414bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:45.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:45.008+0000 7fbebaf77700 1 --2- 172.21.15.3:0/2411570141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe98007370 0x7fbeb414bcd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0007fd0 tx=0x7fbeb000fb90 comp rx=0 tx=0).stop 2023-03-03T06:24:45.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:45.008+0000 7fbebaf77700 1 -- 172.21.15.3:0/2411570141 shutdown_connections 2023-03-03T06:24:45.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:45.008+0000 7fbebaf77700 1 --2- 172.21.15.3:0/2411570141 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbea0004870 0x7fbeb41e6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:45.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:45.008+0000 7fbebaf77700 1 --2- 172.21.15.3:0/2411570141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbea0003a20 0x7fbeb41e6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:45.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:45.008+0000 7fbebaf77700 1 --2- 172.21.15.3:0/2411570141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe98007370 0x7fbeb414bcd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:45.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:45.008+0000 7fbebaf77700 1 -- 172.21.15.3:0/2411570141 >> 172.21.15.3:0/2411570141 conn(0x7fbeb40607c0 msgr2=0x7fbeb415e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:45.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:45.008+0000 7fbebaf77700 1 -- 172.21.15.3:0/2411570141 shutdown_connections 2023-03-03T06:24:45.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:45.008+0000 7fbebaf77700 1 -- 172.21.15.3:0/2411570141 wait complete. 2023-03-03T06:24:45.019 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2023-03-03T06:24:45.049 INFO:tasks.ceph:check_enable_crimson: False 2023-03-03T06:24:45.049 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-03-03T06:24:45.049 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:45.050 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-03-03T06:24:45.094 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:45.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-03-03T06:24:45.150 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:45.143+0000 7f76c27c12c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-03T06:24:45.154 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:45.148+0000 7fd2c67da2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-03T06:24:45.180 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:45.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-03-03T06:24:45.264 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:24:45.264 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-03-03T06:24:45.348 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:45.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-03-03T06:24:45.394 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:45.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-03-03T06:24:45.477 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:45.477 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-03-03T06:24:45.560 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:24:45.560 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-03-03T06:24:45.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd new 51e6c897-995b-4faf-bfb2-9e749ab1c989 0 2023-03-03T06:24:45.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.871+0000 7f8670d43700 1 Processor -- start 2023-03-03T06:24:45.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.871+0000 7f8670d43700 1 -- start start 2023-03-03T06:24:45.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.875+0000 7f8670d43700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 0x7f866c0cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:45.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.875+0000 7f8670d43700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866c0c9e10 con 0x7f866c0b12d0 2023-03-03T06:24:45.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.875+0000 7f8670d43700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866c06ea60 con 0x7f866c06ff00 2023-03-03T06:24:45.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.875+0000 7f8670d43700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c15a850 con 0x7f866c0b51d0 2023-03-03T06:24:45.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.875+0000 7f866bfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f866c06ff00 0x7f866c15e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59468/0 (socket says 172.21.15.32:59468) 2023-03-03T06:24:45.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.875+0000 7f866bfff700 1 -- 172.21.15.32:0/2311992589 learned_addr learned my addr 172.21.15.32:0/2311992589 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-03-03T06:24:45.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.875+0000 7f866b7fe700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 0x7f866c0cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:45.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.875+0000 7f866b7fe700 1 -- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 msgr2=0x7f866c0cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-03-03T06:24:45.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.875+0000 7f866b7fe700 1 -- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 msgr2=0x7f866c0cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:45.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.875+0000 7f866b7fe700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 0x7f866c0cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:45.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.875+0000 7f866b7fe700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 0x7f866c0cc600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:45.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.879+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720844853 0 0) 0x7f866c0c9e10 con 0x7f866c0b12d0 2023-03-03T06:24:45.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.879+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86540035d0 con 0x7f866c0b12d0 2023-03-03T06:24:45.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.879+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344087224 0 0) 0x7f866c06ea60 con 0x7f866c06ff00 2023-03-03T06:24:45.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.879+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f866c0c9e10 con 0x7f866c06ff00 2023-03-03T06:24:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 <== mon.1 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1122448667 0 0) 0x7f866c0c9e10 con 0x7f866c06ff00 2023-03-03T06:24:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 msgr2=0x7f866c0cc600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-03T06:24:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 0x7f866c0cc600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 >> v1:172.21.15.3:6789/0 conn(0x7f866c0b12d0 legacy=0x7f866c15dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:45.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c15edc0 con 0x7f866c06ff00 2023-03-03T06:24:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 <== mon.1 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f8664001670 con 0x7f866c06ff00 2023-03-03T06:24:45.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 >> v1:172.21.15.32:6789/0 conn(0x7f866c06ff00 legacy=0x7f866c15e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8654004c10 0x7f8654005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8654003810 0x7f8654003c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 0x7f865400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:45.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c15a850 con 0x7f8654003810 2023-03-03T06:24:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8654004540 con 0x7f8654004c10 2023-03-03T06:24:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865400bf70 con 0x7f866c0b51d0 2023-03-03T06:24:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866bfff700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 0x7f865400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866b7fe700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8654004c10 0x7f8654005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866bfff700 1 -- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 msgr2=0x7f865400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-03-03T06:24:45.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866bfff700 1 -- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 msgr2=0x7f865400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:45.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866bfff700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 0x7f865400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:45.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866affd700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8654003810 0x7f8654003c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:45.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866bfff700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 0x7f865400ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:45.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866b7fe700 1 -- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 msgr2=0x7f865400ba30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-03T06:24:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866b7fe700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0b51d0 0x7f865400ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866b7fe700 1 -- 172.21.15.32:0/2311992589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8654003810 msgr2=0x7f8654003c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866b7fe700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8654003810 0x7f8654003c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:45.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866b7fe700 1 -- 172.21.15.32:0/2311992589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0c9e10 con 0x7f8654004c10 2023-03-03T06:24:45.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866b7fe700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8654004c10 0x7f8654005000 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f865c009fc0 tx=0x7f86540058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d756696577073f0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:45.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.891+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c007e20 con 0x7f8654004c10 2023-03-03T06:24:45.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.895+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c003a40 con 0x7f8654004c10 2023-03-03T06:24:45.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.895+0000 7f866a7fc700 1 -- 172.21.15.32:0/2311992589 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c003df0 con 0x7f8654004c10 2023-03-03T06:24:45.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.895+0000 7f8670d43700 1 -- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8654004c10 msgr2=0x7f8654005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:45.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.895+0000 7f8670d43700 1 --2- 172.21.15.32:0/2311992589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8654004c10 0x7f8654005000 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f865c009fc0 tx=0x7f86540058a0 comp rx=0 tx=0).stop 2023-03-03T06:24:45.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.895+0000 7f866bfff700 1 -- 172.21.15.32:0/2311992589 reap_dead start 2023-03-03T06:24:45.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.895+0000 7f8670d43700 1 -- 172.21.15.32:0/2311992589 shutdown_connections 2023-03-03T06:24:45.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.895+0000 7f8670d43700 1 -- 172.21.15.32:0/2311992589 >> 172.21.15.32:0/2311992589 conn(0x7f866c069d70 msgr2=0x7f866c06a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:45.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.895+0000 7f8670d43700 1 -- 172.21.15.32:0/2311992589 shutdown_connections 2023-03-03T06:24:45.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.895+0000 7f8670d43700 1 -- 172.21.15.32:0/2311992589 wait complete. 2023-03-03T06:24:45.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.899+0000 7f8670d43700 1 Processor -- start 2023-03-03T06:24:45.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.899+0000 7f8670d43700 1 -- start start 2023-03-03T06:24:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f8670d43700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8654004c10 0x7f866c156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f8670d43700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c0c86c0 0x7f866c0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f8670d43700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0cf830 0x7f866c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f8670d43700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0d01e0 con 0x7f8654004c10 2023-03-03T06:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f8670d43700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0c97f0 con 0x7f866c0c86c0 2023-03-03T06:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f8670d43700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0cd4c0 con 0x7f866c0cf830 2023-03-03T06:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8654004c10 0x7f866c156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0cf830 0x7f866c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8654004c10 0x7f866c156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.32:35336/0 (socket says 172.21.15.32:35336) 2023-03-03T06:24:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866b7fe700 1 -- 172.21.15.32:0/3843553794 learned_addr learned my addr 172.21.15.32:0/3843553794 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-03-03T06:24:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866affd700 1 --2- 172.21.15.32:0/3843553794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c0c86c0 0x7f866c0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866bfff700 1 -- 172.21.15.32:0/3843553794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0cf830 msgr2=0x7f866c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-03-03T06:24:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866bfff700 1 -- 172.21.15.32:0/3843553794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0cf830 msgr2=0x7f866c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-03-03T06:24:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866bfff700 1 --2- 172.21.15.32:0/3843553794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0cf830 0x7f866c0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866bfff700 1 --2- 172.21.15.32:0/3843553794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0cf830 0x7f866c0cfca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866b7fe700 1 -- 172.21.15.32:0/3843553794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0cf830 msgr2=0x7f866c0cfca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866b7fe700 1 --2- 172.21.15.32:0/3843553794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0cf830 0x7f866c0cfca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866b7fe700 1 -- 172.21.15.32:0/3843553794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c0c86c0 msgr2=0x7f866c0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866b7fe700 1 --2- 172.21.15.32:0/3843553794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c0c86c0 0x7f866c0c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.903+0000 7f866b7fe700 1 -- 172.21.15.32:0/3843553794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0ca340 con 0x7f8654004c10 2023-03-03T06:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.907+0000 7f866b7fe700 1 --2- 172.21.15.32:0/3843553794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8654004c10 0x7f866c156330 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f865c0078f0 tx=0x7f865c004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.907+0000 7f8653fff700 1 -- 172.21.15.32:0/3843553794 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c003e00 con 0x7f8654004c10 2023-03-03T06:24:45.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.907+0000 7f8653fff700 1 -- 172.21.15.32:0/3843553794 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c009410 con 0x7f8654004c10 2023-03-03T06:24:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.907+0000 7f8653fff700 1 -- 172.21.15.32:0/3843553794 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c01f440 con 0x7f8654004c10 2023-03-03T06:24:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.907+0000 7f8670d43700 1 -- 172.21.15.32:0/3843553794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866c0ca060 con 0x7f8654004c10 2023-03-03T06:24:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.907+0000 7f8653fff700 1 -- 172.21.15.32:0/3843553794 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f865c012070 con 0x7f8654004c10 2023-03-03T06:24:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.907+0000 7f8670d43700 1 -- 172.21.15.32:0/3843553794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c06e6f0 con 0x7f8654004c10 2023-03-03T06:24:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.907+0000 7f8653fff700 1 -- 172.21.15.32:0/3843553794 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f865c01f8e0 con 0x7f8654004c10 2023-03-03T06:24:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.907+0000 7f8670d43700 1 -- 172.21.15.32:0/3843553794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866c073e90 con 0x7f8654004c10 2023-03-03T06:24:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:45.911+0000 7f8653fff700 1 -- 172.21.15.32:0/3843553794 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f865c047b00 con 0x7f8654004c10 2023-03-03T06:24:45.929 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:45.923+0000 7f76c27c12c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-03T06:24:45.936 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:45.928+0000 7fd2c67da2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-03T06:24:46.037 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:46.028+0000 7fd2c67da2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-03T06:24:46.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.047+0000 7f8670d43700 1 -- 172.21.15.32:0/3843553794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd new", "uuid": "51e6c897-995b-4faf-bfb2-9e749ab1c989", "id": 0} v 0) v1 -- 0x7f866c06dbc0 con 0x7f8654004c10 2023-03-03T06:24:46.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.055+0000 7f8653fff700 1 -- 172.21.15.32:0/3843553794 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "51e6c897-995b-4faf-bfb2-9e749ab1c989", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f865c036070 con 0x7f8654004c10 2023-03-03T06:24:46.062 INFO:teuthology.orchestra.run.smithi032.stdout:0 2023-03-03T06:24:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.059+0000 7f8670d43700 1 -- 172.21.15.32:0/3843553794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8654004c10 msgr2=0x7f866c156330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.059+0000 7f8670d43700 1 --2- 172.21.15.32:0/3843553794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8654004c10 0x7f866c156330 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f865c0078f0 tx=0x7f865c004b60 comp rx=0 tx=0).stop 2023-03-03T06:24:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.059+0000 7f8670d43700 1 -- 172.21.15.32:0/3843553794 shutdown_connections 2023-03-03T06:24:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.059+0000 7f8670d43700 1 --2- 172.21.15.32:0/3843553794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f866c0cf830 0x7f866c0cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.059+0000 7f8670d43700 1 --2- 172.21.15.32:0/3843553794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866c0c86c0 0x7f866c0c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.059+0000 7f8670d43700 1 --2- 172.21.15.32:0/3843553794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8654004c10 0x7f866c156330 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.059+0000 7f8670d43700 1 -- 172.21.15.32:0/3843553794 >> 172.21.15.32:0/3843553794 conn(0x7f866c069d70 msgr2=0x7f866c0af590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.059+0000 7f8670d43700 1 -- 172.21.15.32:0/3843553794 shutdown_connections 2023-03-03T06:24:46.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.059+0000 7f8670d43700 1 -- 172.21.15.32:0/3843553794 wait complete. 2023-03-03T06:24:46.067 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:46.059+0000 7f76c27c12c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-03T06:24:46.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd new b6f1d313-916a-420e-b37f-1f05848d0968 1 2023-03-03T06:24:46.206 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:46.200+0000 7fd2c67da2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-03T06:24:46.244 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:46.239+0000 7f76c27c12c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-03T06:24:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.295+0000 7fadf5984700 1 Processor -- start 2023-03-03T06:24:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.295+0000 7fadf5984700 1 -- start start 2023-03-03T06:24:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.295+0000 7fadf5984700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fadf00cb9a0 0x7fadf00cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.295+0000 7fadf5984700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf015c230 con 0x7fadf006a780 2023-03-03T06:24:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.295+0000 7fadf5984700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf0158b10 con 0x7fadf00709b0 2023-03-03T06:24:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.295+0000 7fadf5984700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf00caac0 con 0x7fadf00cb9a0 2023-03-03T06:24:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.295+0000 7fade7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fadf00cb9a0 0x7fadf00cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.295+0000 7fade7fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fadf00cb9a0 0x7fadf00cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56678/0 (socket says 172.21.15.32:56678) 2023-03-03T06:24:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.295+0000 7fade7fff700 1 -- 172.21.15.32:0/138696228 learned_addr learned my addr 172.21.15.32:0/138696228 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-03-03T06:24:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.295+0000 7fade7fff700 1 -- 172.21.15.32:0/138696228 >> v1:172.21.15.32:6789/0 conn(0x7fadf00709b0 legacy=0x7fadf0163560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.295+0000 7fade7fff700 1 -- 172.21.15.32:0/138696228 >> v1:172.21.15.3:6789/0 conn(0x7fadf006a780 legacy=0x7fadf0158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.295+0000 7fade7fff700 1 -- 172.21.15.32:0/138696228 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf00cd130 con 0x7fadf00cb9a0 2023-03-03T06:24:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.299+0000 7fade7fff700 1 --2- 172.21.15.32:0/138696228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf00cb9a0 0x7fadf00cace0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fade0009910 tx=0x7fade0009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6936bfc46df6b078 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.299+0000 7fade77fe700 1 -- 172.21.15.32:0/138696228 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade000ba60 con 0x7fadf00cb9a0 2023-03-03T06:24:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.299+0000 7fade77fe700 1 -- 172.21.15.32:0/138696228 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade000bbc0 con 0x7fadf00cb9a0 2023-03-03T06:24:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.299+0000 7fade77fe700 1 -- 172.21.15.32:0/138696228 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0009e60 con 0x7fadf00cb9a0 2023-03-03T06:24:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.299+0000 7fadf5984700 1 -- 172.21.15.32:0/138696228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf00cb9a0 msgr2=0x7fadf00cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.299+0000 7fadf5984700 1 --2- 172.21.15.32:0/138696228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf00cb9a0 0x7fadf00cace0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fade0009910 tx=0x7fade0009c20 comp rx=0 tx=0).stop 2023-03-03T06:24:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.299+0000 7fadf5984700 1 -- 172.21.15.32:0/138696228 shutdown_connections 2023-03-03T06:24:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.299+0000 7fadf5984700 1 --2- 172.21.15.32:0/138696228 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf00cb9a0 0x7fadf00cace0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.299+0000 7fadf5984700 1 -- 172.21.15.32:0/138696228 >> 172.21.15.32:0/138696228 conn(0x7fadf00607c0 msgr2=0x7fadf0060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.299+0000 7fadf5984700 1 -- 172.21.15.32:0/138696228 shutdown_connections 2023-03-03T06:24:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.299+0000 7fadf5984700 1 -- 172.21.15.32:0/138696228 wait complete. 2023-03-03T06:24:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf5984700 1 Processor -- start 2023-03-03T06:24:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf5984700 1 -- start start 2023-03-03T06:24:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf5984700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadf006a780 0x7fadf00b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf5984700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf00709b0 0x7fadf00c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf5984700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadf00cb9a0 0x7fadf00c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf5984700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf00ac8d0 con 0x7fadf00cb9a0 2023-03-03T06:24:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf5984700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf00acbb0 con 0x7fadf00709b0 2023-03-03T06:24:46.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf5984700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf00ace90 con 0x7fadf006a780 2023-03-03T06:24:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fade7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf00709b0 0x7fadf00c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fade7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf00709b0 0x7fadf00c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56682/0 (socket says 172.21.15.32:56682) 2023-03-03T06:24:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fade7fff700 1 -- 172.21.15.32:0/477399791 learned_addr learned my addr 172.21.15.32:0/477399791 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-03-03T06:24:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf4982700 1 --2- 172.21.15.32:0/477399791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadf006a780 0x7fadf00b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf4982700 1 -- 172.21.15.32:0/477399791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadf006a780 msgr2=0x7fadf00b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-03-03T06:24:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf4982700 1 -- 172.21.15.32:0/477399791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadf006a780 msgr2=0x7fadf00b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-03-03T06:24:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf4982700 1 --2- 172.21.15.32:0/477399791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadf006a780 0x7fadf00b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf5183700 1 --2- 172.21.15.32:0/477399791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadf00cb9a0 0x7fadf00c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fadf4982700 1 --2- 172.21.15.32:0/477399791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadf006a780 0x7fadf00b4280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fade7fff700 1 -- 172.21.15.32:0/477399791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadf006a780 msgr2=0x7fadf00b4280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:24:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fade7fff700 1 --2- 172.21.15.32:0/477399791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadf006a780 0x7fadf00b4280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fade7fff700 1 -- 172.21.15.32:0/477399791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadf00cb9a0 msgr2=0x7fadf00c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fade7fff700 1 --2- 172.21.15.32:0/477399791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadf00cb9a0 0x7fadf00c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.303+0000 7fade7fff700 1 -- 172.21.15.32:0/477399791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0009570 con 0x7fadf00709b0 2023-03-03T06:24:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.307+0000 7fade7fff700 1 --2- 172.21.15.32:0/477399791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf00709b0 0x7fadf00c8de0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fade0014040 tx=0x7fade0004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.307+0000 7fade57fa700 1 -- 172.21.15.32:0/477399791 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade00038c0 con 0x7fadf00709b0 2023-03-03T06:24:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.307+0000 7fade57fa700 1 -- 172.21.15.32:0/477399791 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade001f070 con 0x7fadf00709b0 2023-03-03T06:24:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.307+0000 7fade57fa700 1 -- 172.21.15.32:0/477399791 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0003c60 con 0x7fadf00709b0 2023-03-03T06:24:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.307+0000 7fadf5984700 1 -- 172.21.15.32:0/477399791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf015c4f0 con 0x7fadf00709b0 2023-03-03T06:24:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.307+0000 7fadf5984700 1 -- 172.21.15.32:0/477399791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf00cbf60 con 0x7fadf00709b0 2023-03-03T06:24:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.307+0000 7fade57fa700 1 -- 172.21.15.32:0/477399791 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fade000f050 con 0x7fadf00709b0 2023-03-03T06:24:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.307+0000 7fade57fa700 1 -- 172.21.15.32:0/477399791 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fade0021070 con 0x7fadf00709b0 2023-03-03T06:24:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.307+0000 7fadf5984700 1 -- 172.21.15.32:0/477399791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf015c7d0 con 0x7fadf00709b0 2023-03-03T06:24:46.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.311+0000 7fade57fa700 1 -- 172.21.15.32:0/477399791 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fade002b020 con 0x7fadf00709b0 2023-03-03T06:24:46.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.391+0000 7fadf5984700 1 -- 172.21.15.32:0/477399791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b6f1d313-916a-420e-b37f-1f05848d0968", "id": 1} v 0) v1 -- 0x7fadf0071250 con 0x7fadf00709b0 2023-03-03T06:24:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.403+0000 7fade57fa700 1 -- 172.21.15.32:0/477399791 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b6f1d313-916a-420e-b37f-1f05848d0968", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fade0015020 con 0x7fadf00709b0 2023-03-03T06:24:46.408 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-03-03T06:24:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.403+0000 7fadf5984700 1 -- 172.21.15.32:0/477399791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf00709b0 msgr2=0x7fadf00c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.403+0000 7fadf5984700 1 --2- 172.21.15.32:0/477399791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf00709b0 0x7fadf00c8de0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fade0014040 tx=0x7fade0004770 comp rx=0 tx=0).stop 2023-03-03T06:24:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.407+0000 7fadf5984700 1 -- 172.21.15.32:0/477399791 shutdown_connections 2023-03-03T06:24:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.407+0000 7fadf5984700 1 --2- 172.21.15.32:0/477399791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadf00cb9a0 0x7fadf00c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.407+0000 7fadf5984700 1 --2- 172.21.15.32:0/477399791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadf00709b0 0x7fadf00c8de0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.407+0000 7fadf5984700 1 --2- 172.21.15.32:0/477399791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadf006a780 0x7fadf00b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.407+0000 7fadf5984700 1 -- 172.21.15.32:0/477399791 >> 172.21.15.32:0/477399791 conn(0x7fadf00607c0 msgr2=0x7fadf0060b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.407+0000 7fadf5984700 1 -- 172.21.15.32:0/477399791 shutdown_connections 2023-03-03T06:24:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.407+0000 7fadf5984700 1 -- 172.21.15.32:0/477399791 wait complete. 2023-03-03T06:24:46.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd new 71d83c5c-1ae5-4441-a1de-d260cc415bcf 2 2023-03-03T06:24:46.572 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:46.564+0000 7fd2c67da2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-03T06:24:46.590 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:46.583+0000 7f76c27c12c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-03T06:24:46.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.639+0000 7fa5829a8700 1 Processor -- start 2023-03-03T06:24:46.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.639+0000 7fa5829a8700 1 -- start start 2023-03-03T06:24:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.639+0000 7fa5829a8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa57c0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.639+0000 7fa5829a8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57c15c230 con 0x7fa57c0710c0 2023-03-03T06:24:46.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.639+0000 7fa5829a8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57c0cd130 con 0x7fa57c0cb9a0 2023-03-03T06:24:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.639+0000 7fa5829a8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0caac0 con 0x7fa57c06a780 2023-03-03T06:24:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.639+0000 7fa5819a6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa57c0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.639+0000 7fa5819a6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa57c0cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40580/0 (socket says 172.21.15.32:40580) 2023-03-03T06:24:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.639+0000 7fa5819a6700 1 -- 172.21.15.32:0/3267826586 learned_addr learned my addr 172.21.15.32:0/3267826586 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-03-03T06:24:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.639+0000 7fa5819a6700 1 -- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 msgr2=0x7fa57c0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-03-03T06:24:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.639+0000 7fa5819a6700 1 -- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 msgr2=0x7fa57c0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:46.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.639+0000 7fa5819a6700 1 --2- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa57c0cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:46.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.639+0000 7fa5819a6700 1 --2- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa57c0cace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:46.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493443168 0 0) 0x7fa57c15c230 con 0x7fa57c0710c0 2023-03-03T06:24:46.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5680057e0 con 0x7fa57c0710c0 2023-03-03T06:24:46.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417414125 0 0) 0x7fa57c0cd130 con 0x7fa57c0cb9a0 2023-03-03T06:24:46.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa57c15c230 con 0x7fa57c0cb9a0 2023-03-03T06:24:46.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3188217656 0 0) 0x7fa5680057e0 con 0x7fa57c0710c0 2023-03-03T06:24:46.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 msgr2=0x7fa57c0cace0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-03T06:24:46.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 --2- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa57c0cace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 >> v1:172.21.15.32:6789/0 conn(0x7fa57c0cb9a0 legacy=0x7fa57c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:46.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c0cbf60 con 0x7fa57c0710c0 2023-03-03T06:24:46.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7fa5740019f0 con 0x7fa57c0710c0 2023-03-03T06:24:46.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 >> v1:172.21.15.3:6789/0 conn(0x7fa57c0710c0 legacy=0x7fa57c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:46.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 --2- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa568005df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:46.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 --2- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa568006350 0x7fa568006a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:46.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 --2- 172.21.15.32:0/3267826586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa568006fc0 0x7fa568007430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:46.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0caac0 con 0x7fa568006fc0 2023-03-03T06:24:46.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56800ddd0 con 0x7fa568006350 2023-03-03T06:24:46.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56800df90 con 0x7fa57c06a780 2023-03-03T06:24:46.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5819a6700 1 --2- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa568005df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:46.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5811a5700 1 --2- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa568006350 0x7fa568006a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:46.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5821a7700 1 --2- 172.21.15.32:0/3267826586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa568006fc0 0x7fa568007430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:46.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5819a6700 1 -- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa568006350 msgr2=0x7fa568006a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:46.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5819a6700 1 --2- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa568006350 0x7fa568006a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5819a6700 1 -- 172.21.15.32:0/3267826586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa568006fc0 msgr2=0x7fa568007430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:46.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5819a6700 1 --2- 172.21.15.32:0/3267826586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa568006fc0 0x7fa568007430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5819a6700 1 -- 172.21.15.32:0/3267826586 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5680057e0 con 0x7fa57c06a780 2023-03-03T06:24:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5819a6700 1 --2- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa568005df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa578009950 tx=0x7fa568007ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98e1988b2247e330 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57800c070 con 0x7fa57c06a780 2023-03-03T06:24:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa578004020 con 0x7fa57c06a780 2023-03-03T06:24:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.643+0000 7fa5809a4700 1 -- 172.21.15.32:0/3267826586 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5780043b0 con 0x7fa57c06a780 2023-03-03T06:24:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.647+0000 7fa5829a8700 1 -- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 msgr2=0x7fa568005df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.647+0000 7fa5829a8700 1 --2- 172.21.15.32:0/3267826586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa568005df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa578009950 tx=0x7fa568007ab0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.647+0000 7fa5821a7700 1 -- 172.21.15.32:0/3267826586 reap_dead start 2023-03-03T06:24:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.647+0000 7fa5829a8700 1 -- 172.21.15.32:0/3267826586 shutdown_connections 2023-03-03T06:24:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.647+0000 7fa5829a8700 1 -- 172.21.15.32:0/3267826586 >> 172.21.15.32:0/3267826586 conn(0x7fa57c0607c0 msgr2=0x7fa57c060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.647+0000 7fa5829a8700 1 -- 172.21.15.32:0/3267826586 shutdown_connections 2023-03-03T06:24:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.647+0000 7fa5829a8700 1 -- 172.21.15.32:0/3267826586 wait complete. 2023-03-03T06:24:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5829a8700 1 Processor -- start 2023-03-03T06:24:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5829a8700 1 -- start start 2023-03-03T06:24:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5829a8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa57c14dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5829a8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c0b16a0 0x7fa57c14e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5829a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b1fb0 0x7fa57c0b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5829a8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0b2960 con 0x7fa57c0b16a0 2023-03-03T06:24:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5829a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0c7780 con 0x7fa57c0b1fb0 2023-03-03T06:24:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5829a8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c14e160 con 0x7fa57c06a780 2023-03-03T06:24:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5819a6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa57c14dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5819a6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa57c14dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40608/0 (socket says 172.21.15.32:40608) 2023-03-03T06:24:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5819a6700 1 -- 172.21.15.32:0/2076528016 learned_addr learned my addr 172.21.15.32:0/2076528016 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-03-03T06:24:46.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5819a6700 1 -- 172.21.15.32:0/2076528016 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 msgr2=0x7fa57c14dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-03-03T06:24:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5819a6700 1 -- 172.21.15.32:0/2076528016 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 msgr2=0x7fa57c14dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-03-03T06:24:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5821a7700 1 --2- 172.21.15.32:0/2076528016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b1fb0 0x7fa57c0b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5819a6700 1 --2- 172.21.15.32:0/2076528016 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa57c14dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5819a6700 1 --2- 172.21.15.32:0/2076528016 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa57c14dc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5811a5700 1 --2- 172.21.15.32:0/2076528016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c0b16a0 0x7fa57c14e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5821a7700 1 -- 172.21.15.32:0/2076528016 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 msgr2=0x7fa57c14dc20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:24:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5821a7700 1 --2- 172.21.15.32:0/2076528016 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa57c14dc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5821a7700 1 -- 172.21.15.32:0/2076528016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c0b16a0 msgr2=0x7fa57c14e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5821a7700 1 --2- 172.21.15.32:0/2076528016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c0b16a0 0x7fa57c14e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5821a7700 1 -- 172.21.15.32:0/2076528016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c15c7d0 con 0x7fa57c0b1fb0 2023-03-03T06:24:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.651+0000 7fa5821a7700 1 --2- 172.21.15.32:0/2076528016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b1fb0 0x7fa57c0b2420 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0b0900 tx=0x7fa574001ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.655+0000 7fa5727fc700 1 -- 172.21.15.32:0/2076528016 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574003c30 con 0x7fa57c0b1fb0 2023-03-03T06:24:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.655+0000 7fa5727fc700 1 -- 172.21.15.32:0/2076528016 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa574003db0 con 0x7fa57c0b1fb0 2023-03-03T06:24:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.655+0000 7fa5727fc700 1 -- 172.21.15.32:0/2076528016 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57400c9b0 con 0x7fa57c0b1fb0 2023-03-03T06:24:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.655+0000 7fa5829a8700 1 -- 172.21.15.32:0/2076528016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57c15c4f0 con 0x7fa57c0b1fb0 2023-03-03T06:24:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.655+0000 7fa5829a8700 1 -- 172.21.15.32:0/2076528016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57c15cab0 con 0x7fa57c0b1fb0 2023-03-03T06:24:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.655+0000 7fa5727fc700 1 -- 172.21.15.32:0/2076528016 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fa5740050d0 con 0x7fa57c0b1fb0 2023-03-03T06:24:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.655+0000 7fa5727fc700 1 -- 172.21.15.32:0/2076528016 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fa574005780 con 0x7fa57c0b1fb0 2023-03-03T06:24:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.655+0000 7fa5829a8700 1 -- 172.21.15.32:0/2076528016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57c073e90 con 0x7fa57c0b1fb0 2023-03-03T06:24:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.659+0000 7fa5727fc700 1 -- 172.21.15.32:0/2076528016 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fa574005360 con 0x7fa57c0b1fb0 2023-03-03T06:24:46.673 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:46.664+0000 7fd2c67da2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-03T06:24:46.694 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:46.687+0000 7f76c27c12c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-03T06:24:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.795+0000 7fa5829a8700 1 -- 172.21.15.32:0/2076528016 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "71d83c5c-1ae5-4441-a1de-d260cc415bcf", "id": 2} v 0) v1 -- 0x7fa57c0ced90 con 0x7fa57c0b1fb0 2023-03-03T06:24:46.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.807+0000 7fa5727fc700 1 -- 172.21.15.32:0/2076528016 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "71d83c5c-1ae5-4441-a1de-d260cc415bcf", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa5740055e0 con 0x7fa57c0b1fb0 2023-03-03T06:24:46.814 INFO:teuthology.orchestra.run.smithi032.stdout:2 2023-03-03T06:24:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.811+0000 7fa5829a8700 1 -- 172.21.15.32:0/2076528016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b1fb0 msgr2=0x7fa57c0b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:46.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.811+0000 7fa5829a8700 1 --2- 172.21.15.32:0/2076528016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b1fb0 0x7fa57c0b2420 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0b0900 tx=0x7fa574001ad0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.811+0000 7fa5829a8700 1 -- 172.21.15.32:0/2076528016 shutdown_connections 2023-03-03T06:24:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.811+0000 7fa5829a8700 1 --2- 172.21.15.32:0/2076528016 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa57c0b1fb0 0x7fa57c0b2420 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.811+0000 7fa5829a8700 1 --2- 172.21.15.32:0/2076528016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c0b16a0 0x7fa57c14e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.811+0000 7fa5829a8700 1 --2- 172.21.15.32:0/2076528016 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa57c06a780 0x7fa57c14dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.811+0000 7fa5829a8700 1 -- 172.21.15.32:0/2076528016 >> 172.21.15.32:0/2076528016 conn(0x7fa57c0607c0 msgr2=0x7fa57c0709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.811+0000 7fa5829a8700 1 -- 172.21.15.32:0/2076528016 shutdown_connections 2023-03-03T06:24:46.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:46.811+0000 7fa5829a8700 1 -- 172.21.15.32:0/2076528016 wait complete. 2023-03-03T06:24:46.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd new b4cc8247-7d5e-4807-971c-28d6b7d7b128 3 2023-03-03T06:24:46.974 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:46.968+0000 7fd2c67da2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-03T06:24:47.038 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:47.031+0000 7f76c27c12c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-03T06:24:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.031+0000 7f04471b4700 1 Processor -- start 2023-03-03T06:24:47.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.031+0000 7f04471b4700 1 -- start start 2023-03-03T06:24:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04471b4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f044015d720 0x7f04401608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04471b4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04400cf610 0x7f04400cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:47.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04471b4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04401600f0 con 0x7f0440156cc0 2023-03-03T06:24:47.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04471b4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04400cff70 con 0x7f044015d720 2023-03-03T06:24:47.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04471b4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04400cd420 con 0x7f04400cf610 2023-03-03T06:24:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04469b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04400cf610 0x7f04400cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04469b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04400cf610 0x7f04400cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40618/0 (socket says 172.21.15.32:40618) 2023-03-03T06:24:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04469b3700 1 -- 172.21.15.32:0/808378330 learned_addr learned my addr 172.21.15.32:0/808378330 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-03-03T06:24:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04459b1700 1 --2- 172.21.15.32:0/808378330 >> v2:172.21.15.32:3300/0 conn(0x7f044015d720 0x7f04401608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04469b3700 1 -- 172.21.15.32:0/808378330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04400cf610 msgr2=0x7f04400cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-03-03T06:24:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04469b3700 1 -- 172.21.15.32:0/808378330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04400cf610 msgr2=0x7f04400cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04469b3700 1 --2- 172.21.15.32:0/808378330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04400cf610 0x7f04400cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04469b3700 1 --2- 172.21.15.32:0/808378330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04400cf610 0x7f04400cfa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04459b1700 1 -- 172.21.15.32:0/808378330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04400cf610 msgr2=0x7f04400cfa30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-03T06:24:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04459b1700 1 --2- 172.21.15.32:0/808378330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04400cf610 0x7f04400cfa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04459b1700 1 -- 172.21.15.32:0/808378330 >> v1:172.21.15.32:6789/0 conn(0x7f0440156cc0 legacy=0x7f04400713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.035+0000 7f04459b1700 1 -- 172.21.15.32:0/808378330 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04400cd700 con 0x7f044015d720 2023-03-03T06:24:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.039+0000 7f04459b1700 1 --2- 172.21.15.32:0/808378330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f044015d720 0x7f04401608b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0430009850 tx=0x7f0430009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=644e094ded530808 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.039+0000 7f04451b0700 1 -- 172.21.15.32:0/808378330 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430011830 con 0x7f044015d720 2023-03-03T06:24:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.039+0000 7f04451b0700 1 -- 172.21.15.32:0/808378330 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0430011990 con 0x7f044015d720 2023-03-03T06:24:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.039+0000 7f04451b0700 1 -- 172.21.15.32:0/808378330 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430011d20 con 0x7f044015d720 2023-03-03T06:24:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.039+0000 7f04471b4700 1 -- 172.21.15.32:0/808378330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f044015d720 msgr2=0x7f04401608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.039+0000 7f04471b4700 1 --2- 172.21.15.32:0/808378330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f044015d720 0x7f04401608b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0430009850 tx=0x7f0430009b60 comp rx=0 tx=0).stop 2023-03-03T06:24:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.039+0000 7f04471b4700 1 -- 172.21.15.32:0/808378330 shutdown_connections 2023-03-03T06:24:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.039+0000 7f04471b4700 1 --2- 172.21.15.32:0/808378330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f04400cf610 0x7f04400cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.039+0000 7f04471b4700 1 --2- 172.21.15.32:0/808378330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f044015d720 0x7f04401608b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.039+0000 7f04471b4700 1 -- 172.21.15.32:0/808378330 >> 172.21.15.32:0/808378330 conn(0x7f044006e5c0 msgr2=0x7f044006e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.039+0000 7f04471b4700 1 -- 172.21.15.32:0/808378330 shutdown_connections 2023-03-03T06:24:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.039+0000 7f04471b4700 1 -- 172.21.15.32:0/808378330 wait complete. 2023-03-03T06:24:47.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.043+0000 7f04471b4700 1 Processor -- start 2023-03-03T06:24:47.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.043+0000 7f04471b4700 1 -- start start 2023-03-03T06:24:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04471b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04400cf610 0x7f044014c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04471b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0440156cc0 0x7f04400cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04471b4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f044015d720 0x7f04400cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04471b4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044014cf30 con 0x7f04400cf610 2023-03-03T06:24:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04471b4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04400cc5d0 con 0x7f0440156cc0 2023-03-03T06:24:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04471b4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04400cbd60 con 0x7f044015d720 2023-03-03T06:24:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04459b1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0440156cc0 0x7f04400cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04469b3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f044015d720 0x7f04400cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04459b1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0440156cc0 0x7f04400cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56712/0 (socket says 172.21.15.32:56712) 2023-03-03T06:24:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04459b1700 1 -- 172.21.15.32:0/1968508891 learned_addr learned my addr 172.21.15.32:0/1968508891 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-03-03T06:24:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04461b2700 1 --2- 172.21.15.32:0/1968508891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04400cf610 0x7f044014c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04469b3700 1 -- 172.21.15.32:0/1968508891 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f044015d720 msgr2=0x7f04400cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-03-03T06:24:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04469b3700 1 -- 172.21.15.32:0/1968508891 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f044015d720 msgr2=0x7f04400cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-03-03T06:24:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04469b3700 1 --2- 172.21.15.32:0/1968508891 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f044015d720 0x7f04400cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04469b3700 1 --2- 172.21.15.32:0/1968508891 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f044015d720 0x7f04400cb820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04459b1700 1 -- 172.21.15.32:0/1968508891 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f044015d720 msgr2=0x7f04400cb820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:24:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04459b1700 1 --2- 172.21.15.32:0/1968508891 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f044015d720 0x7f04400cb820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04459b1700 1 -- 172.21.15.32:0/1968508891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04400cf610 msgr2=0x7f044014c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04459b1700 1 --2- 172.21.15.32:0/1968508891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04400cf610 0x7f044014c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04459b1700 1 -- 172.21.15.32:0/1968508891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04300094b0 con 0x7f0440156cc0 2023-03-03T06:24:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04459b1700 1 --2- 172.21.15.32:0/1968508891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0440156cc0 0x7f04400cb2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0430014040 tx=0x7f0430004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f0436ffd700 1 -- 172.21.15.32:0/1968508891 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04300036c0 con 0x7f0440156cc0 2023-03-03T06:24:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f0436ffd700 1 -- 172.21.15.32:0/1968508891 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043001e070 con 0x7f0440156cc0 2023-03-03T06:24:47.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f0436ffd700 1 -- 172.21.15.32:0/1968508891 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430003a60 con 0x7f0440156cc0 2023-03-03T06:24:47.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04471b4700 1 -- 172.21.15.32:0/1968508891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04400cd9e0 con 0x7f0440156cc0 2023-03-03T06:24:47.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f04471b4700 1 -- 172.21.15.32:0/1968508891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0440160390 con 0x7f0440156cc0 2023-03-03T06:24:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f0436ffd700 1 -- 172.21.15.32:0/1968508891 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f043000f050 con 0x7f0440156cc0 2023-03-03T06:24:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.047+0000 7f0436ffd700 1 -- 172.21.15.32:0/1968508891 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f043002dca0 con 0x7f0440156cc0 2023-03-03T06:24:47.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.051+0000 7f04471b4700 1 -- 172.21.15.32:0/1968508891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0440160df0 con 0x7f0440156cc0 2023-03-03T06:24:47.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.055+0000 7f0436ffd700 1 -- 172.21.15.32:0/1968508891 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f0430044b00 con 0x7f0440156cc0 2023-03-03T06:24:47.090 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:47.084+0000 7fd2c67da2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-03T06:24:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.131+0000 7f04471b4700 1 -- 172.21.15.32:0/1968508891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b4cc8247-7d5e-4807-971c-28d6b7d7b128", "id": 3} v 0) v1 -- 0x7f0440073e90 con 0x7f0440156cc0 2023-03-03T06:24:47.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.143+0000 7f0436ffd700 1 -- 172.21.15.32:0/1968508891 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b4cc8247-7d5e-4807-971c-28d6b7d7b128", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0430033070 con 0x7f0440156cc0 2023-03-03T06:24:47.150 INFO:teuthology.orchestra.run.smithi032.stdout:3 2023-03-03T06:24:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.143+0000 7f04471b4700 1 -- 172.21.15.32:0/1968508891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0440156cc0 msgr2=0x7f04400cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.143+0000 7f04471b4700 1 --2- 172.21.15.32:0/1968508891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0440156cc0 0x7f04400cb2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0430014040 tx=0x7f0430004770 comp rx=0 tx=0).stop 2023-03-03T06:24:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.147+0000 7f04471b4700 1 -- 172.21.15.32:0/1968508891 shutdown_connections 2023-03-03T06:24:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.147+0000 7f04471b4700 1 --2- 172.21.15.32:0/1968508891 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f044015d720 0x7f04400cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.147+0000 7f04471b4700 1 --2- 172.21.15.32:0/1968508891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0440156cc0 0x7f04400cb2e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.147+0000 7f04471b4700 1 --2- 172.21.15.32:0/1968508891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04400cf610 0x7f044014c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.147+0000 7f04471b4700 1 -- 172.21.15.32:0/1968508891 >> 172.21.15.32:0/1968508891 conn(0x7f044006e5c0 msgr2=0x7f0440067b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.147+0000 7f04471b4700 1 -- 172.21.15.32:0/1968508891 shutdown_connections 2023-03-03T06:24:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.147+0000 7f04471b4700 1 -- 172.21.15.32:0/1968508891 wait complete. 2023-03-03T06:24:47.154 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:47.147+0000 7f76c27c12c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-03T06:24:47.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd new 3aea1cb1-e900-42d7-906a-cc8de12f00c1 4 2023-03-03T06:24:47.209 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:47.200+0000 7fd2c67da2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-03T06:24:47.271 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:47.263+0000 7f76c27c12c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-03T06:24:47.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.363+0000 7fc3653ce700 1 Processor -- start 2023-03-03T06:24:47.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.363+0000 7fc3653ce700 1 -- start start 2023-03-03T06:24:47.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc3653ce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3600b19e0 0x7fc3600cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc3653ce700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36006e6f0 con 0x7fc3601609b0 2023-03-03T06:24:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc3653ce700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36015d880 con 0x7fc3600cc9b0 2023-03-03T06:24:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc35ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc3600cc9b0 0x7fc3600b12d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59510/0 (socket says 172.21.15.32:59510) 2023-03-03T06:24:47.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc35ffff700 1 -- 172.21.15.32:0/3137430372 learned_addr learned my addr 172.21.15.32:0/3137430372 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-03-03T06:24:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc3653ce700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc360160f50 con 0x7fc3600b19e0 2023-03-03T06:24:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc35f7fe700 1 --2- 172.21.15.32:0/3137430372 >> v2:172.21.15.32:3300/0 conn(0x7fc3600b19e0 0x7fc3600cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc35effd700 1 -- 172.21.15.32:0/3137430372 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384999851 0 0) 0x7fc36015d880 con 0x7fc3600cc9b0 2023-03-03T06:24:47.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc35effd700 1 -- 172.21.15.32:0/3137430372 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3480035d0 con 0x7fc3600cc9b0 2023-03-03T06:24:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc35f7fe700 1 -- 172.21.15.32:0/3137430372 >> v1:172.21.15.32:6789/0 conn(0x7fc3600cc9b0 legacy=0x7fc3600b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc35f7fe700 1 -- 172.21.15.32:0/3137430372 >> v1:172.21.15.3:6789/0 conn(0x7fc3601609b0 legacy=0x7fc36006eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc35f7fe700 1 -- 172.21.15.32:0/3137430372 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3600b20c0 con 0x7fc3600b19e0 2023-03-03T06:24:47.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc35f7fe700 1 --2- 172.21.15.32:0/3137430372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3600b19e0 0x7fc3600cf0b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc354009850 tx=0x7fc354009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cd0d905136e233 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc35effd700 1 -- 172.21.15.32:0/3137430372 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354011830 con 0x7fc3600b19e0 2023-03-03T06:24:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc35effd700 1 -- 172.21.15.32:0/3137430372 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc354011990 con 0x7fc3600b19e0 2023-03-03T06:24:47.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.367+0000 7fc35effd700 1 -- 172.21.15.32:0/3137430372 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354011d40 con 0x7fc3600b19e0 2023-03-03T06:24:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.371+0000 7fc3653ce700 1 -- 172.21.15.32:0/3137430372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3600b19e0 msgr2=0x7fc3600cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.371+0000 7fc3653ce700 1 --2- 172.21.15.32:0/3137430372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3600b19e0 0x7fc3600cf0b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc354009850 tx=0x7fc354009b60 comp rx=0 tx=0).stop 2023-03-03T06:24:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.371+0000 7fc3653ce700 1 -- 172.21.15.32:0/3137430372 shutdown_connections 2023-03-03T06:24:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.371+0000 7fc3653ce700 1 --2- 172.21.15.32:0/3137430372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3600b19e0 0x7fc3600cf0b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.371+0000 7fc3653ce700 1 -- 172.21.15.32:0/3137430372 >> 172.21.15.32:0/3137430372 conn(0x7fc360069d70 msgr2=0x7fc36006a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:47.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.371+0000 7fc3653ce700 1 -- 172.21.15.32:0/3137430372 shutdown_connections 2023-03-03T06:24:47.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.371+0000 7fc3653ce700 1 -- 172.21.15.32:0/3137430372 wait complete. 2023-03-03T06:24:47.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.371+0000 7fc3653ce700 1 Processor -- start 2023-03-03T06:24:47.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc3653ce700 1 -- start start 2023-03-03T06:24:47.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc3653ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3600b19e0 0x7fc3601597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc3653ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3600cc9b0 0x7fc360159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc3653ce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3601609b0 0x7fc3600ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc3653ce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3600b3a40 con 0x7fc3600b19e0 2023-03-03T06:24:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc3653ce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3600b3760 con 0x7fc3600cc9b0 2023-03-03T06:24:47.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc3653ce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36015a270 con 0x7fc3601609b0 2023-03-03T06:24:47.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc35f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3600cc9b0 0x7fc360159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc364bcd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3601609b0 0x7fc3600ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc35ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3600b19e0 0x7fc3601597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc35f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3600cc9b0 0x7fc360159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56728/0 (socket says 172.21.15.32:56728) 2023-03-03T06:24:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc35f7fe700 1 -- 172.21.15.32:0/3362768315 learned_addr learned my addr 172.21.15.32:0/3362768315 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-03-03T06:24:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc364bcd700 1 -- 172.21.15.32:0/3362768315 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3601609b0 msgr2=0x7fc3600ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-03-03T06:24:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc364bcd700 1 -- 172.21.15.32:0/3362768315 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3601609b0 msgr2=0x7fc3600ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-03-03T06:24:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc364bcd700 1 --2- 172.21.15.32:0/3362768315 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3601609b0 0x7fc3600ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc364bcd700 1 --2- 172.21.15.32:0/3362768315 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3601609b0 0x7fc3600ae0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc35f7fe700 1 -- 172.21.15.32:0/3362768315 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3601609b0 msgr2=0x7fc3600ae0c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:24:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc35f7fe700 1 --2- 172.21.15.32:0/3362768315 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3601609b0 0x7fc3600ae0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc35f7fe700 1 -- 172.21.15.32:0/3362768315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3600b19e0 msgr2=0x7fc3601597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc35f7fe700 1 --2- 172.21.15.32:0/3362768315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3600b19e0 0x7fc3601597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.375+0000 7fc35f7fe700 1 -- 172.21.15.32:0/3362768315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3540094b0 con 0x7fc3600cc9b0 2023-03-03T06:24:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.379+0000 7fc35f7fe700 1 --2- 172.21.15.32:0/3362768315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3600cc9b0 0x7fc360159d30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc354000c00 tx=0x7fc354004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.379+0000 7fc35cff9700 1 -- 172.21.15.32:0/3362768315 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354019a00 con 0x7fc3600cc9b0 2023-03-03T06:24:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.379+0000 7fc35cff9700 1 -- 172.21.15.32:0/3362768315 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35401e070 con 0x7fc3600cc9b0 2023-03-03T06:24:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.379+0000 7fc35cff9700 1 -- 172.21.15.32:0/3362768315 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354019da0 con 0x7fc3600cc9b0 2023-03-03T06:24:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.379+0000 7fc3653ce700 1 -- 172.21.15.32:0/3362768315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3600cd810 con 0x7fc3600cc9b0 2023-03-03T06:24:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.379+0000 7fc3653ce700 1 -- 172.21.15.32:0/3362768315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3600cd530 con 0x7fc3600cc9b0 2023-03-03T06:24:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.379+0000 7fc35cff9700 1 -- 172.21.15.32:0/3362768315 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc35400f050 con 0x7fc3600cc9b0 2023-03-03T06:24:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.379+0000 7fc35cff9700 1 -- 172.21.15.32:0/3362768315 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fc35402dc70 con 0x7fc3600cc9b0 2023-03-03T06:24:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.379+0000 7fc3653ce700 1 -- 172.21.15.32:0/3362768315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3600cdaf0 con 0x7fc3600cc9b0 2023-03-03T06:24:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.383+0000 7fc35cff9700 1 -- 172.21.15.32:0/3362768315 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fc354020070 con 0x7fc3600cc9b0 2023-03-03T06:24:47.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.463+0000 7fc3653ce700 1 -- 172.21.15.32:0/3362768315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3aea1cb1-e900-42d7-906a-cc8de12f00c1", "id": 4} v 0) v1 -- 0x7fc3600aefd0 con 0x7fc3600cc9b0 2023-03-03T06:24:47.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.475+0000 7fc35cff9700 1 -- 172.21.15.32:0/3362768315 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3aea1cb1-e900-42d7-906a-cc8de12f00c1", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc354015020 con 0x7fc3600cc9b0 2023-03-03T06:24:47.480 INFO:teuthology.orchestra.run.smithi032.stdout:4 2023-03-03T06:24:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.475+0000 7fc3653ce700 1 -- 172.21.15.32:0/3362768315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3600cc9b0 msgr2=0x7fc360159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.475+0000 7fc3653ce700 1 --2- 172.21.15.32:0/3362768315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3600cc9b0 0x7fc360159d30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc354000c00 tx=0x7fc354004770 comp rx=0 tx=0).stop 2023-03-03T06:24:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.475+0000 7fc3653ce700 1 -- 172.21.15.32:0/3362768315 shutdown_connections 2023-03-03T06:24:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.475+0000 7fc3653ce700 1 --2- 172.21.15.32:0/3362768315 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3601609b0 0x7fc3600ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.475+0000 7fc3653ce700 1 --2- 172.21.15.32:0/3362768315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3600cc9b0 0x7fc360159d30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.475+0000 7fc3653ce700 1 --2- 172.21.15.32:0/3362768315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3600b19e0 0x7fc3601597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.475+0000 7fc3653ce700 1 -- 172.21.15.32:0/3362768315 >> 172.21.15.32:0/3362768315 conn(0x7fc360069d70 msgr2=0x7fc36015e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.475+0000 7fc3653ce700 1 -- 172.21.15.32:0/3362768315 shutdown_connections 2023-03-03T06:24:47.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.475+0000 7fc3653ce700 1 -- 172.21.15.32:0/3362768315 wait complete. 2023-03-03T06:24:47.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd new a7e1f2d6-bdce-403f-a0af-726bca7bca68 5 2023-03-03T06:24:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.695+0000 7f10731a1700 1 Processor -- start 2023-03-03T06:24:47.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.695+0000 7f10731a1700 1 -- start start 2023-03-03T06:24:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.695+0000 7f10731a1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f106c0d0b30 0x7f106c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.695+0000 7f10731a1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106c0d14d0 con 0x7f106c156ba0 2023-03-03T06:24:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.695+0000 7f10731a1700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106c0cf3b0 con 0x7f106c15d6e0 2023-03-03T06:24:47.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.695+0000 7f10731a1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106c156f70 con 0x7f106c0d0b30 2023-03-03T06:24:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f10729a0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f106c0d0b30 0x7f106c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f107219f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f106c156ba0 0x7f106c071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.32:49056/0 (socket says 172.21.15.32:49056) 2023-03-03T06:24:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f107219f700 1 -- 172.21.15.32:0/3976961373 learned_addr learned my addr 172.21.15.32:0/3976961373 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-03-03T06:24:47.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f107199e700 1 --1- 172.21.15.32:0/3976961373 >> v1:172.21.15.32:6789/0 conn(0x7f106c15d6e0 0x7f106c0cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59512/0 (socket says 172.21.15.32:59512) 2023-03-03T06:24:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f107119d700 1 -- 172.21.15.32:0/3976961373 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140839136 0 0) 0x7f106c0d14d0 con 0x7f106c156ba0 2023-03-03T06:24:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f107119d700 1 -- 172.21.15.32:0/3976961373 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10580035d0 con 0x7f106c156ba0 2023-03-03T06:24:47.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f107119d700 1 -- 172.21.15.32:0/3976961373 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114061979 0 0) 0x7f106c0cf3b0 con 0x7f106c15d6e0 2023-03-03T06:24:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f107119d700 1 -- 172.21.15.32:0/3976961373 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f106c0d14d0 con 0x7f106c15d6e0 2023-03-03T06:24:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f10729a0700 1 -- 172.21.15.32:0/3976961373 >> v1:172.21.15.32:6789/0 conn(0x7f106c15d6e0 legacy=0x7f106c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f10729a0700 1 -- 172.21.15.32:0/3976961373 >> v1:172.21.15.3:6789/0 conn(0x7f106c156ba0 legacy=0x7f106c071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f10729a0700 1 -- 172.21.15.32:0/3976961373 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c0cd320 con 0x7f106c0d0b30 2023-03-03T06:24:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f10729a0700 1 --2- 172.21.15.32:0/3976961373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106c0d0b30 0x7f106c0d0f90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1064009d90 tx=0x7f106400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e955ab7d49884a9c server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f107119d700 1 -- 172.21.15.32:0/3976961373 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106400e070 con 0x7f106c0d0b30 2023-03-03T06:24:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f107119d700 1 -- 172.21.15.32:0/3976961373 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106400bb20 con 0x7f106c0d0b30 2023-03-03T06:24:47.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f107119d700 1 -- 172.21.15.32:0/3976961373 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064012430 con 0x7f106c0d0b30 2023-03-03T06:24:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f10731a1700 1 -- 172.21.15.32:0/3976961373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106c0d0b30 msgr2=0x7f106c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.699+0000 7f10731a1700 1 --2- 172.21.15.32:0/3976961373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106c0d0b30 0x7f106c0d0f90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1064009d90 tx=0x7f106400b330 comp rx=0 tx=0).stop 2023-03-03T06:24:47.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.703+0000 7f10731a1700 1 -- 172.21.15.32:0/3976961373 shutdown_connections 2023-03-03T06:24:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.703+0000 7f10731a1700 1 --2- 172.21.15.32:0/3976961373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106c0d0b30 0x7f106c0d0f90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.703+0000 7f10731a1700 1 -- 172.21.15.32:0/3976961373 >> 172.21.15.32:0/3976961373 conn(0x7f106c06e5c0 msgr2=0x7f106c06e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.703+0000 7f10731a1700 1 -- 172.21.15.32:0/3976961373 shutdown_connections 2023-03-03T06:24:47.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.703+0000 7f10731a1700 1 -- 172.21.15.32:0/3976961373 wait complete. 2023-03-03T06:24:47.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.703+0000 7f10731a1700 1 Processor -- start 2023-03-03T06:24:47.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.703+0000 7f10731a1700 1 -- start start 2023-03-03T06:24:47.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f10731a1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106c0d0b30 0x7f106c0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:47.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f10731a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f106c156ba0 0x7f106c1517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:47.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f10731a1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f106c15d6e0 0x7f106c0b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f10731a1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0c7de0 con 0x7f106c0d0b30 2023-03-03T06:24:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f10731a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0c80c0 con 0x7f106c156ba0 2023-03-03T06:24:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f10731a1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0c83a0 con 0x7f106c15d6e0 2023-03-03T06:24:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f107199e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f106c156ba0 0x7f106c1517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f10729a0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f106c15d6e0 0x7f106c0b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f107199e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f106c156ba0 0x7f106c1517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56732/0 (socket says 172.21.15.32:56732) 2023-03-03T06:24:47.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f107199e700 1 -- 172.21.15.32:0/2869147153 learned_addr learned my addr 172.21.15.32:0/2869147153 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-03-03T06:24:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f107219f700 1 --2- 172.21.15.32:0/2869147153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106c0d0b30 0x7f106c0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f10729a0700 1 -- 172.21.15.32:0/2869147153 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f106c15d6e0 msgr2=0x7f106c0b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-03-03T06:24:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f10729a0700 1 -- 172.21.15.32:0/2869147153 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f106c15d6e0 msgr2=0x7f106c0b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-03-03T06:24:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f10729a0700 1 --2- 172.21.15.32:0/2869147153 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f106c15d6e0 0x7f106c0b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f10729a0700 1 --2- 172.21.15.32:0/2869147153 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f106c15d6e0 0x7f106c0b3600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:47.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f107199e700 1 -- 172.21.15.32:0/2869147153 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f106c15d6e0 msgr2=0x7f106c0b3600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:24:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f107199e700 1 --2- 172.21.15.32:0/2869147153 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f106c15d6e0 0x7f106c0b3600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f107199e700 1 -- 172.21.15.32:0/2869147153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106c0d0b30 msgr2=0x7f106c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f107199e700 1 --2- 172.21.15.32:0/2869147153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106c0d0b30 0x7f106c0c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f107199e700 1 -- 172.21.15.32:0/2869147153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1064009a70 con 0x7f106c156ba0 2023-03-03T06:24:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f107199e700 1 --2- 172.21.15.32:0/2869147153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f106c156ba0 0x7f106c1517d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c700 tx=0x7f105c00ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f1062ffd700 1 -- 172.21.15.32:0/2869147153 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c005cd0 con 0x7f106c156ba0 2023-03-03T06:24:47.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f1062ffd700 1 -- 172.21.15.32:0/2869147153 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c005e30 con 0x7f106c156ba0 2023-03-03T06:24:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f10731a1700 1 -- 172.21.15.32:0/2869147153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106c0cd8e0 con 0x7f106c156ba0 2023-03-03T06:24:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f1062ffd700 1 -- 172.21.15.32:0/2869147153 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c0072f0 con 0x7f106c156ba0 2023-03-03T06:24:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.707+0000 7f10731a1700 1 -- 172.21.15.32:0/2869147153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106c0cd600 con 0x7f106c156ba0 2023-03-03T06:24:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.711+0000 7f1062ffd700 1 -- 172.21.15.32:0/2869147153 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f105c019030 con 0x7f106c156ba0 2023-03-03T06:24:47.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.711+0000 7f1062ffd700 1 -- 172.21.15.32:0/2869147153 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f105c01c440 con 0x7f106c156ba0 2023-03-03T06:24:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.711+0000 7f10731a1700 1 -- 172.21.15.32:0/2869147153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106c0cff50 con 0x7f106c156ba0 2023-03-03T06:24:47.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.715+0000 7f1062ffd700 1 -- 172.21.15.32:0/2869147153 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f105c017070 con 0x7f106c156ba0 2023-03-03T06:24:47.777 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:47.768+0000 7fd2c67da2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-03T06:24:47.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.791+0000 7f10731a1700 1 -- 172.21.15.32:0/2869147153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a7e1f2d6-bdce-403f-a0af-726bca7bca68", "id": 5} v 0) v1 -- 0x7f106c073e90 con 0x7f106c156ba0 2023-03-03T06:24:47.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.803+0000 7f1062ffd700 1 -- 172.21.15.32:0/2869147153 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a7e1f2d6-bdce-403f-a0af-726bca7bca68", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f105c007450 con 0x7f106c156ba0 2023-03-03T06:24:47.809 INFO:teuthology.orchestra.run.smithi032.stdout:5 2023-03-03T06:24:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.803+0000 7f10731a1700 1 -- 172.21.15.32:0/2869147153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f106c156ba0 msgr2=0x7f106c1517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:47.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.803+0000 7f10731a1700 1 --2- 172.21.15.32:0/2869147153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f106c156ba0 0x7f106c1517d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c700 tx=0x7f105c00ca10 comp rx=0 tx=0).stop 2023-03-03T06:24:47.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.803+0000 7f10731a1700 1 -- 172.21.15.32:0/2869147153 shutdown_connections 2023-03-03T06:24:47.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.803+0000 7f10731a1700 1 --2- 172.21.15.32:0/2869147153 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f106c15d6e0 0x7f106c0b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.803+0000 7f10731a1700 1 --2- 172.21.15.32:0/2869147153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f106c156ba0 0x7f106c1517d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.803+0000 7f10731a1700 1 --2- 172.21.15.32:0/2869147153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f106c0d0b30 0x7f106c0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:47.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.803+0000 7f10731a1700 1 -- 172.21.15.32:0/2869147153 >> 172.21.15.32:0/2869147153 conn(0x7f106c06e5c0 msgr2=0x7f106c071280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:47.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.803+0000 7f10731a1700 1 -- 172.21.15.32:0/2869147153 shutdown_connections 2023-03-03T06:24:47.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:47.803+0000 7f10731a1700 1 -- 172.21.15.32:0/2869147153 wait complete. 2023-03-03T06:24:47.823 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:47.815+0000 7f76c27c12c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-03T06:24:47.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd new c395a30c-360d-408e-93b9-2819033f7725 6 2023-03-03T06:24:48.013 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:48.008+0000 7fd2c67da2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-03T06:24:48.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.006+0000 7f57694b8700 1 Processor -- start 2023-03-03T06:24:48.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f57694b8700 1 -- start start 2023-03-03T06:24:48.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f57694b8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576406a780 0x7f57640cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f57694b8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576415c230 con 0x7f57640710c0 2023-03-03T06:24:48.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f57694b8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57640cd130 con 0x7f57640cb9a0 2023-03-03T06:24:48.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f57694b8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57640caac0 con 0x7f576406a780 2023-03-03T06:24:48.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f57637fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f57640cb9a0 0x7f57640709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59514/0 (socket says 172.21.15.32:59514) 2023-03-03T06:24:48.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f57637fe700 1 -- 172.21.15.32:0/389902309 learned_addr learned my addr 172.21.15.32:0/389902309 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-03-03T06:24:48.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5763fff700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576406a780 0x7f57640cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514577614 0 0) 0x7f57640cd130 con 0x7f57640cb9a0 2023-03-03T06:24:48.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f574c0034f0 con 0x7f57640cb9a0 2023-03-03T06:24:48.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5763fff700 1 -- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576406a780 msgr2=0x7f57640cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-03-03T06:24:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5763fff700 1 -- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576406a780 msgr2=0x7f57640cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5763fff700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576406a780 0x7f57640cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 <== mon.1 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 615109546 0 0) 0x7f574c0034f0 con 0x7f57640cb9a0 2023-03-03T06:24:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576406a780 msgr2=0x7f57640cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5763fff700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576406a780 0x7f57640cace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576406a780 0x7f57640cace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 >> v1:172.21.15.3:6789/0 conn(0x7f57640710c0 legacy=0x7f57640ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:48.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57640cbf60 con 0x7f57640cb9a0 2023-03-03T06:24:48.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 <== mon.1 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f5758001670 con 0x7f57640cb9a0 2023-03-03T06:24:48.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 >> v1:172.21.15.32:6789/0 conn(0x7f57640cb9a0 legacy=0x7f57640709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:48.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574c003b40 0x7f574c003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576406a780 0x7f574c00a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574c00b100 0x7f574c00d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640caac0 con 0x7f574c003b40 2023-03-03T06:24:48.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c00dd60 con 0x7f574c00b100 2023-03-03T06:24:48.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.010+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c004760 con 0x7f576406a780 2023-03-03T06:24:48.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f5768cb7700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574c00b100 0x7f574c00d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f5763fff700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574c003b40 0x7f574c003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f57637fe700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576406a780 0x7f574c00a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f5768cb7700 1 -- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576406a780 msgr2=0x7f574c00a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:48.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f5768cb7700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576406a780 0x7f574c00a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f5768cb7700 1 -- 172.21.15.32:0/389902309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574c003b40 msgr2=0x7f574c003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f5768cb7700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574c003b40 0x7f574c003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f5768cb7700 1 -- 172.21.15.32:0/389902309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c0034f0 con 0x7f574c00b100 2023-03-03T06:24:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f5768cb7700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574c00b100 0x7f574c00d580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f575c00c160 tx=0x7f574c0037a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=952dbf6c082ffdc8 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c001db0 con 0x7f574c00b100 2023-03-03T06:24:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c00cca0 con 0x7f574c00b100 2023-03-03T06:24:48.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f5762ffd700 1 -- 172.21.15.32:0/389902309 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c003c60 con 0x7f574c00b100 2023-03-03T06:24:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f57694b8700 1 -- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574c00b100 msgr2=0x7f574c00d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f57694b8700 1 --2- 172.21.15.32:0/389902309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f574c00b100 0x7f574c00d580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f575c00c160 tx=0x7f574c0037a0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f5768cb7700 1 -- 172.21.15.32:0/389902309 reap_dead start 2023-03-03T06:24:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f57694b8700 1 -- 172.21.15.32:0/389902309 shutdown_connections 2023-03-03T06:24:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.014+0000 7f57694b8700 1 -- 172.21.15.32:0/389902309 >> 172.21.15.32:0/389902309 conn(0x7f57640607c0 msgr2=0x7f5764060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.018+0000 7f57694b8700 1 -- 172.21.15.32:0/389902309 shutdown_connections 2023-03-03T06:24:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.018+0000 7f57694b8700 1 -- 172.21.15.32:0/389902309 wait complete. 2023-03-03T06:24:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57694b8700 1 Processor -- start 2023-03-03T06:24:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57694b8700 1 -- start start 2023-03-03T06:24:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57694b8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574c00b100 0x7f5764152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57694b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f576415b640 0x7f57640b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57694b8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576415ba90 0x7f57640b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57694b8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640b16a0 con 0x7f574c00b100 2023-03-03T06:24:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57694b8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640b18b0 con 0x7f576415b640 2023-03-03T06:24:48.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57694b8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57640b1b90 con 0x7f576415ba90 2023-03-03T06:24:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f5768cb7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576415ba90 0x7f57640b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f576415b640 0x7f57640b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f5768cb7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576415ba90 0x7f57640b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40650/0 (socket says 172.21.15.32:40650) 2023-03-03T06:24:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f5768cb7700 1 -- 172.21.15.32:0/511225434 learned_addr learned my addr 172.21.15.32:0/511225434 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-03-03T06:24:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f5768cb7700 1 -- 172.21.15.32:0/511225434 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576415ba90 msgr2=0x7f57640b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-03-03T06:24:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f5768cb7700 1 -- 172.21.15.32:0/511225434 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576415ba90 msgr2=0x7f57640b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-03-03T06:24:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f5763fff700 1 --2- 172.21.15.32:0/511225434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574c00b100 0x7f5764152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f5768cb7700 1 --2- 172.21.15.32:0/511225434 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576415ba90 0x7f57640b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:48.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f5768cb7700 1 --2- 172.21.15.32:0/511225434 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576415ba90 0x7f57640b3780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57637fe700 1 -- 172.21.15.32:0/511225434 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576415ba90 msgr2=0x7f57640b3780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:24:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57637fe700 1 --2- 172.21.15.32:0/511225434 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576415ba90 0x7f57640b3780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57637fe700 1 -- 172.21.15.32:0/511225434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574c00b100 msgr2=0x7f5764152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57637fe700 1 --2- 172.21.15.32:0/511225434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574c00b100 0x7f5764152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.022+0000 7f57637fe700 1 -- 172.21.15.32:0/511225434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576415c7d0 con 0x7f576415b640 2023-03-03T06:24:48.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.026+0000 7f57637fe700 1 --2- 172.21.15.32:0/511225434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f576415b640 0x7f57640b3240 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f574c0045d0 tx=0x7f575800edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:48.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.026+0000 7f5760ff9700 1 -- 172.21.15.32:0/511225434 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758005e00 con 0x7f576415b640 2023-03-03T06:24:48.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.026+0000 7f5760ff9700 1 -- 172.21.15.32:0/511225434 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575800c7e0 con 0x7f576415b640 2023-03-03T06:24:48.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.026+0000 7f5760ff9700 1 -- 172.21.15.32:0/511225434 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575800cb90 con 0x7f576415b640 2023-03-03T06:24:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.026+0000 7f57694b8700 1 -- 172.21.15.32:0/511225434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576415c4f0 con 0x7f576415b640 2023-03-03T06:24:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.026+0000 7f57694b8700 1 -- 172.21.15.32:0/511225434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576415cab0 con 0x7f576415b640 2023-03-03T06:24:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.026+0000 7f5760ff9700 1 -- 172.21.15.32:0/511225434 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f575801d460 con 0x7f576415b640 2023-03-03T06:24:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.026+0000 7f5760ff9700 1 -- 172.21.15.32:0/511225434 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f575801dce0 con 0x7f576415b640 2023-03-03T06:24:48.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.026+0000 7f57694b8700 1 -- 172.21.15.32:0/511225434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57640610f0 con 0x7f576415b640 2023-03-03T06:24:48.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.030+0000 7f5760ff9700 1 -- 172.21.15.32:0/511225434 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f575801d7a0 con 0x7f576415b640 2023-03-03T06:24:48.049 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:48.042+0000 7f76c27c12c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-03T06:24:48.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.110+0000 7f57694b8700 1 -- 172.21.15.32:0/511225434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c395a30c-360d-408e-93b9-2819033f7725", "id": 6} v 0) v1 -- 0x7f5764073e90 con 0x7f576415b640 2023-03-03T06:24:48.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.118+0000 7f5760ff9700 1 -- 172.21.15.32:0/511225434 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c395a30c-360d-408e-93b9-2819033f7725", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f575801d940 con 0x7f576415b640 2023-03-03T06:24:48.127 INFO:teuthology.orchestra.run.smithi032.stdout:6 2023-03-03T06:24:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.122+0000 7f57694b8700 1 -- 172.21.15.32:0/511225434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f576415b640 msgr2=0x7f57640b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.122+0000 7f57694b8700 1 --2- 172.21.15.32:0/511225434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f576415b640 0x7f57640b3240 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f574c0045d0 tx=0x7f575800edd0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.122+0000 7f57694b8700 1 -- 172.21.15.32:0/511225434 shutdown_connections 2023-03-03T06:24:48.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.122+0000 7f57694b8700 1 --2- 172.21.15.32:0/511225434 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f576415ba90 0x7f57640b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.122+0000 7f57694b8700 1 --2- 172.21.15.32:0/511225434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f576415b640 0x7f57640b3240 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.122+0000 7f57694b8700 1 --2- 172.21.15.32:0/511225434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f574c00b100 0x7f5764152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.122+0000 7f57694b8700 1 -- 172.21.15.32:0/511225434 >> 172.21.15.32:0/511225434 conn(0x7f57640607c0 msgr2=0x7f5764153ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:48.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.122+0000 7f57694b8700 1 -- 172.21.15.32:0/511225434 shutdown_connections 2023-03-03T06:24:48.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.122+0000 7f57694b8700 1 -- 172.21.15.32:0/511225434 wait complete. 2023-03-03T06:24:48.160 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd new db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d 7 2023-03-03T06:24:48.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.350+0000 7f974759e700 1 Processor -- start 2023-03-03T06:24:48.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.350+0000 7f974759e700 1 -- start start 2023-03-03T06:24:48.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.350+0000 7f974759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9748071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.350+0000 7f974759e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f974815d6e0 0x7f97480cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.350+0000 7f974759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97480cf3b0 con 0x7f97480cfd80 2023-03-03T06:24:48.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.350+0000 7f974759e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9748156f70 con 0x7f974815d6e0 2023-03-03T06:24:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.350+0000 7f974759e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97480cd320 con 0x7f9748156ba0 2023-03-03T06:24:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.350+0000 7f9746d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f97480cfd80 0x7f97480d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59524/0 (socket says 172.21.15.32:59524) 2023-03-03T06:24:48.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.350+0000 7f9746d9d700 1 -- 172.21.15.32:0/1218543613 learned_addr learned my addr 172.21.15.32:0/1218543613 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-03-03T06:24:48.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974659c700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9748071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f9745d9b700 1 --2- 172.21.15.32:0/1218543613 >> v2:172.21.15.3:3300/0 conn(0x7f974815d6e0 0x7f97480cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974659c700 1 -- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 msgr2=0x7f9748071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-03-03T06:24:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170364320 0 0) 0x7f97480cf3b0 con 0x7f97480cfd80 2023-03-03T06:24:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974659c700 1 -- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 msgr2=0x7f9748071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:48.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974659c700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9748071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97300035d0 con 0x7f97480cfd80 2023-03-03T06:24:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974659c700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9748071280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:48.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 <== mon.1 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2155903282 0 0) 0x7f97300035d0 con 0x7f97480cfd80 2023-03-03T06:24:48.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 msgr2=0x7f9748071280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-03T06:24:48.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9748071280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 >> v2:172.21.15.3:3300/0 conn(0x7f974815d6e0 msgr2=0x7f97480cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 --2- 172.21.15.32:0/1218543613 >> v2:172.21.15.3:3300/0 conn(0x7f974815d6e0 0x7f97480cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97480cd600 con 0x7f97480cfd80 2023-03-03T06:24:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 <== mon.1 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f9740001670 con 0x7f97480cfd80 2023-03-03T06:24:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 >> v1:172.21.15.32:6789/0 conn(0x7f97480cfd80 legacy=0x7f97480d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9730003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9730003fa0 0x7f9730004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9730004db0 0x7f9730005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9748156f70 con 0x7f9730004db0 2023-03-03T06:24:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97480cd320 con 0x7f9730003fa0 2023-03-03T06:24:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973000ba60 con 0x7f9748156ba0 2023-03-03T06:24:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f9745d9b700 1 --2- 172.21.15.32:0/1218543613 >> v2:172.21.15.3:3300/0 conn(0x7f974815d6e0 0x7f97480cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:24:48.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974659c700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9730003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f9745d9b700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9730003fa0 0x7f9730004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f9746d9d700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9730004db0 0x7f9730005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974659c700 1 -- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9730003fa0 msgr2=0x7f9730004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:48.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974659c700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9730003fa0 0x7f9730004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974659c700 1 -- 172.21.15.32:0/1218543613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9730004db0 msgr2=0x7f9730005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974659c700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9730004db0 0x7f9730005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974659c700 1 -- 172.21.15.32:0/1218543613 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97300035d0 con 0x7f9748156ba0 2023-03-03T06:24:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f9746d9d700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9730004db0 0x7f9730005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:24:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974659c700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9730003a20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9738009fd0 tx=0x7f97300058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6df6d675194b141 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:48.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.354+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738007e20 con 0x7f9748156ba0 2023-03-03T06:24:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.358+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9738003a40 con 0x7f9748156ba0 2023-03-03T06:24:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.358+0000 7f974559a700 1 -- 172.21.15.32:0/1218543613 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738003df0 con 0x7f9748156ba0 2023-03-03T06:24:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.358+0000 7f974759e700 1 -- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 msgr2=0x7f9730003a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.358+0000 7f974759e700 1 --2- 172.21.15.32:0/1218543613 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9730003a20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9738009fd0 tx=0x7f97300058a0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.358+0000 7f9746d9d700 1 -- 172.21.15.32:0/1218543613 reap_dead start 2023-03-03T06:24:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.358+0000 7f974759e700 1 -- 172.21.15.32:0/1218543613 shutdown_connections 2023-03-03T06:24:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.358+0000 7f974759e700 1 -- 172.21.15.32:0/1218543613 >> 172.21.15.32:0/1218543613 conn(0x7f974806e5c0 msgr2=0x7f974806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.358+0000 7f974759e700 1 -- 172.21.15.32:0/1218543613 shutdown_connections 2023-03-03T06:24:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.358+0000 7f974759e700 1 -- 172.21.15.32:0/1218543613 wait complete. 2023-03-03T06:24:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.362+0000 7f974759e700 1 Processor -- start 2023-03-03T06:24:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.362+0000 7f974759e700 1 -- start start 2023-03-03T06:24:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.362+0000 7f974759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9748157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.362+0000 7f974759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97481590d0 0x7f97480c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.362+0000 7f974759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974814ab40 0x7f974814afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.362+0000 7f974759e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9748158690 con 0x7f97481590d0 2023-03-03T06:24:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.362+0000 7f974759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974814b4f0 con 0x7f974814ab40 2023-03-03T06:24:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.362+0000 7f974759e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9748157a10 con 0x7f9748156ba0 2023-03-03T06:24:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.362+0000 7f974659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9748157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9746d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974814ab40 0x7f974814afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f974659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9748157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.32:40688/0 (socket says 172.21.15.32:40688) 2023-03-03T06:24:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f974659c700 1 -- 172.21.15.32:0/2191136901 learned_addr learned my addr 172.21.15.32:0/2191136901 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-03-03T06:24:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9745d9b700 1 --2- 172.21.15.32:0/2191136901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97481590d0 0x7f97480c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f974659c700 1 -- 172.21.15.32:0/2191136901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 msgr2=0x7f9748157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-03-03T06:24:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f974659c700 1 -- 172.21.15.32:0/2191136901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 msgr2=0x7f9748157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-03-03T06:24:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f974659c700 1 --2- 172.21.15.32:0/2191136901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9748157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f974659c700 1 --2- 172.21.15.32:0/2191136901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9748157110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9746d9d700 1 -- 172.21.15.32:0/2191136901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 msgr2=0x7f9748157110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:24:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9746d9d700 1 --2- 172.21.15.32:0/2191136901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9748157110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9746d9d700 1 -- 172.21.15.32:0/2191136901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97481590d0 msgr2=0x7f97480c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9746d9d700 1 --2- 172.21.15.32:0/2191136901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97481590d0 0x7f97480c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9746d9d700 1 -- 172.21.15.32:0/2191136901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97480cef40 con 0x7f974814ab40 2023-03-03T06:24:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9745d9b700 1 --2- 172.21.15.32:0/2191136901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97481590d0 0x7f97480c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:24:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9746d9d700 1 --2- 172.21.15.32:0/2191136901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974814ab40 0x7f974814afb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f97480d2b20 tx=0x7f974000db00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:48.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9736ffd700 1 -- 172.21.15.32:0/2191136901 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740004ca0 con 0x7f974814ab40 2023-03-03T06:24:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9736ffd700 1 -- 172.21.15.32:0/2191136901 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9740004e00 con 0x7f974814ab40 2023-03-03T06:24:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f974759e700 1 -- 172.21.15.32:0/2191136901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97480cd8e0 con 0x7f974814ab40 2023-03-03T06:24:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9736ffd700 1 -- 172.21.15.32:0/2191136901 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97400072b0 con 0x7f974814ab40 2023-03-03T06:24:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f974759e700 1 -- 172.21.15.32:0/2191136901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97480cf0d0 con 0x7f974814ab40 2023-03-03T06:24:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9736ffd700 1 -- 172.21.15.32:0/2191136901 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f974001c070 con 0x7f974814ab40 2023-03-03T06:24:48.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.366+0000 7f9736ffd700 1 -- 172.21.15.32:0/2191136901 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f9740007700 con 0x7f974814ab40 2023-03-03T06:24:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.370+0000 7f974759e700 1 -- 172.21.15.32:0/2191136901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97481587d0 con 0x7f974814ab40 2023-03-03T06:24:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.370+0000 7f9736ffd700 1 -- 172.21.15.32:0/2191136901 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f97481587d0 con 0x7f974814ab40 2023-03-03T06:24:48.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.450+0000 7f974759e700 1 -- 172.21.15.32:0/2191136901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d", "id": 7} v 0) v1 -- 0x7f9748073e90 con 0x7f974814ab40 2023-03-03T06:24:48.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.462+0000 7f9736ffd700 1 -- 172.21.15.32:0/2191136901 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f974001a060 con 0x7f974814ab40 2023-03-03T06:24:48.471 INFO:teuthology.orchestra.run.smithi032.stdout:7 2023-03-03T06:24:48.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.466+0000 7f974759e700 1 -- 172.21.15.32:0/2191136901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974814ab40 msgr2=0x7f974814afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.466+0000 7f974759e700 1 --2- 172.21.15.32:0/2191136901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974814ab40 0x7f974814afb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f97480d2b20 tx=0x7f974000db00 comp rx=0 tx=0).stop 2023-03-03T06:24:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.466+0000 7f974759e700 1 -- 172.21.15.32:0/2191136901 shutdown_connections 2023-03-03T06:24:48.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.466+0000 7f974759e700 1 --2- 172.21.15.32:0/2191136901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f974814ab40 0x7f974814afb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.466+0000 7f974759e700 1 --2- 172.21.15.32:0/2191136901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97481590d0 0x7f97480c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.466+0000 7f974759e700 1 --2- 172.21.15.32:0/2191136901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9748156ba0 0x7f9748157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.466+0000 7f974759e700 1 -- 172.21.15.32:0/2191136901 >> 172.21.15.32:0/2191136901 conn(0x7f974806e5c0 msgr2=0x7f974815ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.466+0000 7f974759e700 1 -- 172.21.15.32:0/2191136901 shutdown_connections 2023-03-03T06:24:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-03-03T06:24:48.466+0000 7f974759e700 1 -- 172.21.15.32:0/2191136901 wait complete. 2023-03-03T06:24:48.503 INFO:tasks.ceph.osd.0:Restarting daemon 2023-03-03T06:24:48.503 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-03-03T06:24:48.510 INFO:tasks.ceph.osd.0:Started 2023-03-03T06:24:48.510 INFO:tasks.ceph.osd.1:Restarting daemon 2023-03-03T06:24:48.510 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-03-03T06:24:48.515 INFO:tasks.ceph.osd.1:Started 2023-03-03T06:24:48.516 INFO:tasks.ceph.osd.2:Restarting daemon 2023-03-03T06:24:48.516 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-03-03T06:24:48.520 INFO:tasks.ceph.osd.2:Started 2023-03-03T06:24:48.521 INFO:tasks.ceph.osd.3:Restarting daemon 2023-03-03T06:24:48.521 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-03-03T06:24:48.528 INFO:tasks.ceph.osd.3:Started 2023-03-03T06:24:48.529 INFO:tasks.ceph.osd.4:Restarting daemon 2023-03-03T06:24:48.529 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-03-03T06:24:48.536 INFO:tasks.ceph.osd.4:Started 2023-03-03T06:24:48.536 INFO:tasks.ceph.osd.5:Restarting daemon 2023-03-03T06:24:48.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-03-03T06:24:48.542 INFO:tasks.ceph.osd.5:Started 2023-03-03T06:24:48.543 INFO:tasks.ceph.osd.6:Restarting daemon 2023-03-03T06:24:48.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-03-03T06:24:48.548 INFO:tasks.ceph.osd.6:Started 2023-03-03T06:24:48.549 INFO:tasks.ceph.osd.7:Restarting daemon 2023-03-03T06:24:48.549 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-03-03T06:24:48.556 INFO:tasks.ceph.osd.7:Started 2023-03-03T06:24:48.557 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:24:48.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.792+0000 7f8932fce700 1 Processor -- start 2023-03-03T06:24:48.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.792+0000 7f8932fce700 1 -- start start 2023-03-03T06:24:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f8932fce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f892c0cb1b0 0x7f892c0b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f8932fce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 0x7f892c0ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f8932fce700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892c15e190 con 0x7f892c0cec50 2023-03-03T06:24:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f8932fce700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892c15a7d0 con 0x7f892c0cb1b0 2023-03-03T06:24:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f8932fce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c0b1630 con 0x7f892c0cb580 2023-03-03T06:24:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f89327cd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f892c0cec50 0x7f892c0c9d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42052/0 (socket says 172.21.15.3:42052) 2023-03-03T06:24:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f89327cd700 1 -- 172.21.15.3:0/2814195365 learned_addr learned my addr 172.21.15.3:0/2814195365 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:24:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f8931fcc700 1 --2- 172.21.15.3:0/2814195365 >> v2:172.21.15.32:3300/0 conn(0x7f892c0cb1b0 0x7f892c0b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f89317cb700 1 --2- 172.21.15.3:0/2814195365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 0x7f892c0ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f8930fca700 1 -- 172.21.15.3:0/2814195365 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734271119 0 0) 0x7f892c15e190 con 0x7f892c0cec50 2023-03-03T06:24:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f8930fca700 1 -- 172.21.15.3:0/2814195365 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89180035d0 con 0x7f892c0cec50 2023-03-03T06:24:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f89317cb700 1 -- 172.21.15.3:0/2814195365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 msgr2=0x7f892c0ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-03-03T06:24:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f89317cb700 1 -- 172.21.15.3:0/2814195365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 msgr2=0x7f892c0ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-03T06:24:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f89317cb700 1 --2- 172.21.15.3:0/2814195365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 0x7f892c0ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f89317cb700 1 --2- 172.21.15.3:0/2814195365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 0x7f892c0ce710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f8931fcc700 1 -- 172.21.15.3:0/2814195365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 msgr2=0x7f892c0ce710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-03T06:24:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f8931fcc700 1 --2- 172.21.15.3:0/2814195365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 0x7f892c0ce710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f8931fcc700 1 -- 172.21.15.3:0/2814195365 >> v1:172.21.15.3:6789/0 conn(0x7f892c0cec50 legacy=0x7f892c0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f8931fcc700 1 -- 172.21.15.3:0/2814195365 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c15e470 con 0x7f892c0cb1b0 2023-03-03T06:24:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.796+0000 7f8931fcc700 1 --2- 172.21.15.3:0/2814195365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892c0cb1b0 0x7f892c0b0cd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8928009d90 tx=0x7f892800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d63cc1d704d6f02 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.800+0000 7f8930fca700 1 -- 172.21.15.3:0/2814195365 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892800e070 con 0x7f892c0cb1b0 2023-03-03T06:24:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.800+0000 7f8930fca700 1 -- 172.21.15.3:0/2814195365 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892800bb20 con 0x7f892c0cb1b0 2023-03-03T06:24:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.800+0000 7f8930fca700 1 -- 172.21.15.3:0/2814195365 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8928012430 con 0x7f892c0cb1b0 2023-03-03T06:24:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.800+0000 7f8932fce700 1 -- 172.21.15.3:0/2814195365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892c0cb1b0 msgr2=0x7f892c0b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.800+0000 7f8932fce700 1 --2- 172.21.15.3:0/2814195365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892c0cb1b0 0x7f892c0b0cd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8928009d90 tx=0x7f892800b330 comp rx=0 tx=0).stop 2023-03-03T06:24:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.800+0000 7f8932fce700 1 -- 172.21.15.3:0/2814195365 shutdown_connections 2023-03-03T06:24:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.800+0000 7f8932fce700 1 --2- 172.21.15.3:0/2814195365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 0x7f892c0ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.800+0000 7f8932fce700 1 --2- 172.21.15.3:0/2814195365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892c0cb1b0 0x7f892c0b0cd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.800+0000 7f8932fce700 1 -- 172.21.15.3:0/2814195365 >> 172.21.15.3:0/2814195365 conn(0x7f892c070a30 msgr2=0x7f892c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.800+0000 7f8932fce700 1 -- 172.21.15.3:0/2814195365 shutdown_connections 2023-03-03T06:24:48.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.800+0000 7f8932fce700 1 -- 172.21.15.3:0/2814195365 wait complete. 2023-03-03T06:24:48.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.804+0000 7f8932fce700 1 Processor -- start 2023-03-03T06:24:48.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.804+0000 7f8932fce700 1 -- start start 2023-03-03T06:24:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.804+0000 7f8932fce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892c0cb1b0 0x7f892c1f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.804+0000 7f8932fce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 0x7f892c15c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.804+0000 7f8932fce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f892c0cec50 0x7f892c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.804+0000 7f8932fce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c15d780 con 0x7f892c0cec50 2023-03-03T06:24:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.804+0000 7f8932fce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c1f1a10 con 0x7f892c0cb1b0 2023-03-03T06:24:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.804+0000 7f8932fce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c15cb00 con 0x7f892c0cb580 2023-03-03T06:24:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.804+0000 7f89327cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f892c0cec50 0x7f892c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.804+0000 7f89327cd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f892c0cec50 0x7f892c0c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42740/0 (socket says 172.21.15.3:42740) 2023-03-03T06:24:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89327cd700 1 -- 172.21.15.3:0/2628043608 learned_addr learned my addr 172.21.15.3:0/2628043608 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:24:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f8931fcc700 1 --2- 172.21.15.3:0/2628043608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892c0cb1b0 0x7f892c1f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89317cb700 1 --2- 172.21.15.3:0/2628043608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 0x7f892c15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89317cb700 1 -- 172.21.15.3:0/2628043608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 msgr2=0x7f892c15c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2023-03-03T06:24:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89317cb700 1 -- 172.21.15.3:0/2628043608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 msgr2=0x7f892c15c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-03-03T06:24:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89317cb700 1 --2- 172.21.15.3:0/2628043608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 0x7f892c15c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-03T06:24:48.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89317cb700 1 --2- 172.21.15.3:0/2628043608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 0x7f892c15c090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:24:48.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89327cd700 1 -- 172.21.15.3:0/2628043608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 msgr2=0x7f892c15c090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:24:48.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89327cd700 1 --2- 172.21.15.3:0/2628043608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 0x7f892c15c090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89327cd700 1 -- 172.21.15.3:0/2628043608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892c0cb1b0 msgr2=0x7f892c1f14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:48.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89327cd700 1 --2- 172.21.15.3:0/2628043608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892c0cb1b0 0x7f892c1f14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89327cd700 1 -- 172.21.15.3:0/2628043608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8928009a70 con 0x7f892c0cec50 2023-03-03T06:24:48.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f8931fcc700 1 --2- 172.21.15.3:0/2628043608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892c0cb1b0 0x7f892c1f14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:24:48.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89327cd700 1 --2- 172.21.15.3:0/2628043608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f892c0cec50 0x7f892c0c9d40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f892400c4d0 tx=0x7f892400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:48.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89227fc700 1 -- 172.21.15.3:0/2628043608 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924005d90 con 0x7f892c0cec50 2023-03-03T06:24:48.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89227fc700 1 -- 172.21.15.3:0/2628043608 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89240070d0 con 0x7f892c0cec50 2023-03-03T06:24:48.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89227fc700 1 -- 172.21.15.3:0/2628043608 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924007460 con 0x7f892c0cec50 2023-03-03T06:24:48.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f8932fce700 1 -- 172.21.15.3:0/2628043608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892c15ea30 con 0x7f892c0cec50 2023-03-03T06:24:48.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f8932fce700 1 -- 172.21.15.3:0/2628043608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892c15e750 con 0x7f892c0cec50 2023-03-03T06:24:48.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89227fc700 1 -- 172.21.15.3:0/2628043608 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f8924019030 con 0x7f892c0cec50 2023-03-03T06:24:48.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.808+0000 7f89227fc700 1 -- 172.21.15.3:0/2628043608 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f892401ba80 con 0x7f892c0cec50 2023-03-03T06:24:48.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.812+0000 7f8932fce700 1 -- 172.21.15.3:0/2628043608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892c0af7e0 con 0x7f892c0cec50 2023-03-03T06:24:48.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.816+0000 7f89227fc700 1 -- 172.21.15.3:0/2628043608 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f8924017070 con 0x7f892c0cec50 2023-03-03T06:24:48.878 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:48.870+0000 7f76c27c12c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-03T06:24:48.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.884+0000 7f8932fce700 1 -- 172.21.15.3:0/2628043608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f892c073e90 con 0x7f892c0cec50 2023-03-03T06:24:48.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.888+0000 7f89227fc700 1 -- 172.21.15.3:0/2628043608 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f892401c4f0 con 0x7f892c0cec50 2023-03-03T06:24:48.894 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:24:48.895 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":10,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:24:48.459752+0000","last_up_change":"0.000000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:24:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.888+0000 7f8932fce700 1 -- 172.21.15.3:0/2628043608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f892c0cec50 msgr2=0x7f892c0c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.888+0000 7f8932fce700 1 --2- 172.21.15.3:0/2628043608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f892c0cec50 0x7f892c0c9d40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f892400c4d0 tx=0x7f892400c7e0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.888+0000 7f8932fce700 1 -- 172.21.15.3:0/2628043608 shutdown_connections 2023-03-03T06:24:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.888+0000 7f8932fce700 1 --2- 172.21.15.3:0/2628043608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f892c0cec50 0x7f892c0c9d40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.888+0000 7f8932fce700 1 --2- 172.21.15.3:0/2628043608 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f892c0cb580 0x7f892c15c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.888+0000 7f8932fce700 1 --2- 172.21.15.3:0/2628043608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f892c0cb1b0 0x7f892c1f14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.888+0000 7f8932fce700 1 -- 172.21.15.3:0/2628043608 >> 172.21.15.3:0/2628043608 conn(0x7f892c070a30 msgr2=0x7f892c159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.888+0000 7f8932fce700 1 -- 172.21.15.3:0/2628043608 shutdown_connections 2023-03-03T06:24:48.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:24:48.888+0000 7f8932fce700 1 -- 172.21.15.3:0/2628043608 wait complete. 2023-03-03T06:24:48.930 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-03-03T06:24:48.930 INFO:tasks.ceph:Waiting for OSDs to come up 2023-03-03T06:24:48.931 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:48.920+0000 7fd2c67da2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-03T06:24:49.186 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:49.178+0000 7f76c27c12c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-03T06:24:49.245 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:49.236+0000 7fd2c67da2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-03T06:24:49.245 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-03-03T06:24:49.236+0000 7f3a305ec700 -1 Falling back to public interface 2023-03-03T06:24:49.251 INFO:tasks.ceph.osd.4.smithi032.stderr:2023-03-03T06:24:49.246+0000 7ff939816700 -1 Falling back to public interface 2023-03-03T06:24:49.252 INFO:tasks.ceph.osd.6.smithi032.stderr:2023-03-03T06:24:49.246+0000 7f0420954700 -1 Falling back to public interface 2023-03-03T06:24:49.252 INFO:tasks.ceph.osd.4.smithi032.stderr:2023-03-03T06:24:49.246+0000 7ff9387e1700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6802/0: (98) Address already in use 2023-03-03T06:24:49.253 INFO:tasks.ceph.osd.4.smithi032.stderr:2023-03-03T06:24:49.246+0000 7ff9387e1700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6805/0: (98) Address already in use 2023-03-03T06:24:49.256 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-03-03T06:24:49.248+0000 7f5f1307c700 -1 Falling back to public interface 2023-03-03T06:24:49.271 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-03-03T06:24:49.264+0000 7fbe73cd7700 -1 Falling back to public interface 2023-03-03T06:24:49.284 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-03-03T06:24:49.276+0000 7f69f47c4700 -1 Falling back to public interface 2023-03-03T06:24:49.290 INFO:tasks.ceph.osd.7.smithi032.stderr:2023-03-03T06:24:49.282+0000 7f03d9388700 -1 Falling back to public interface 2023-03-03T06:24:49.303 INFO:tasks.ceph.osd.5.smithi032.stderr:2023-03-03T06:24:49.294+0000 7f0058d71700 -1 Falling back to public interface 2023-03-03T06:24:49.453 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:49.446+0000 7f76c27c12c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-03T06:24:49.532 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:49.524+0000 7fd2c67da2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-03T06:24:49.569 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:49.562+0000 7f76c27c12c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-03T06:24:49.646 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:49.640+0000 7fd2c67da2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-03T06:24:49.737 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-03T06:24:49.916 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.908+0000 7f83ce11a700 1 Processor -- start 2023-03-03T06:24:49.917 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.908+0000 7f83ce11a700 1 -- start start 2023-03-03T06:24:49.918 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83ce11a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f83c815a3f0 0x7f83c80c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:49.919 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83ce11a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c8070a70 con 0x7f83c80ce350 2023-03-03T06:24:49.919 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83cc917700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f83c80ce350 0x7f83c80ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42068/0 (socket says 172.21.15.3:42068) 2023-03-03T06:24:49.919 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83cc917700 1 -- 172.21.15.3:0/1742409852 learned_addr learned my addr 172.21.15.3:0/1742409852 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:24:49.919 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83ce11a700 1 -- 172.21.15.3:0/1742409852 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c8071270 con 0x7f83c815a020 2023-03-03T06:24:49.919 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83ce11a700 1 -- 172.21.15.3:0/1742409852 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c80cd5f0 con 0x7f83c815a3f0 2023-03-03T06:24:49.920 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83cd919700 1 --2- 172.21.15.3:0/1742409852 >> v2:172.21.15.3:3300/0 conn(0x7f83c815a3f0 0x7f83c80c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:49.921 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125212382 0 0) 0x7f83b8001110 con 0x7f83c80ce350 2023-03-03T06:24:49.921 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83b40035d0 con 0x7f83c80ce350 2023-03-03T06:24:49.921 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3106941056 0 0) 0x7f83b40035d0 con 0x7f83c80ce350 2023-03-03T06:24:49.921 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 >> v2:172.21.15.3:3300/0 conn(0x7f83c815a3f0 msgr2=0x7f83c80c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:49.921 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 --2- 172.21.15.3:0/1742409852 >> v2:172.21.15.3:3300/0 conn(0x7f83c815a3f0 0x7f83c80c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:49.921 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 >> v1:172.21.15.32:6789/0 conn(0x7f83c815a020 legacy=0x7f83c8070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:49.922 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c8070cb0 con 0x7f83c80ce350 2023-03-03T06:24:49.922 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f83b8001800 con 0x7f83c80ce350 2023-03-03T06:24:49.922 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 >> v1:172.21.15.3:6789/0 conn(0x7f83c80ce350 legacy=0x7f83c80ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:49.922 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 --2- 172.21.15.3:0/1742409852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4003a20 0x7f83b4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:49.922 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 --2- 172.21.15.3:0/1742409852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4004410 0x7f83b40048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:49.922 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 --2- 172.21.15.3:0/1742409852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83b40050d0 0x7f83b400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:49.922 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c80cd5f0 con 0x7f83b4004410 2023-03-03T06:24:49.922 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b400c020 con 0x7f83b4003a20 2023-03-03T06:24:49.923 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b400c1e0 con 0x7f83b40050d0 2023-03-03T06:24:49.923 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83cc917700 1 --2- 172.21.15.3:0/1742409852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4004410 0x7f83b40048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:49.926 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83cc917700 1 -- 172.21.15.3:0/1742409852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83b40050d0 msgr2=0x7f83b400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:49.926 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83cc917700 1 --2- 172.21.15.3:0/1742409852 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83b40050d0 0x7f83b400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:49.926 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83cc917700 1 -- 172.21.15.3:0/1742409852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4003a20 msgr2=0x7f83b4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:49.926 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.912+0000 7f83cd118700 1 --2- 172.21.15.3:0/1742409852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4003a20 0x7f83b4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:49.926 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83cc917700 1 --2- 172.21.15.3:0/1742409852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4003a20 0x7f83b4003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:49.927 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83cc917700 1 -- 172.21.15.3:0/1742409852 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b40035d0 con 0x7f83b4004410 2023-03-03T06:24:49.927 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83cd118700 1 --2- 172.21.15.3:0/1742409852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83b4003a20 0x7f83b4003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:24:49.927 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83cd919700 1 -- 172.21.15.3:0/1742409852 reap_dead start 2023-03-03T06:24:49.927 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83cc917700 1 --2- 172.21.15.3:0/1742409852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4004410 0x7f83b40048d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f83b800c2f0 tx=0x7f83b40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540fe1ab6d135ae6 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:49.927 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b800cd70 con 0x7f83b4004410 2023-03-03T06:24:49.927 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b8005980 con 0x7f83b4004410 2023-03-03T06:24:49.927 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83bffff700 1 -- 172.21.15.3:0/1742409852 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8005d30 con 0x7f83b4004410 2023-03-03T06:24:49.928 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83ce11a700 1 -- 172.21.15.3:0/1742409852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4004410 msgr2=0x7f83b40048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:49.928 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83ce11a700 1 --2- 172.21.15.3:0/1742409852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4004410 0x7f83b40048d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f83b800c2f0 tx=0x7f83b40058a0 comp rx=0 tx=0).stop 2023-03-03T06:24:49.928 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83ce11a700 1 -- 172.21.15.3:0/1742409852 shutdown_connections 2023-03-03T06:24:49.928 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83ce11a700 1 --2- 172.21.15.3:0/1742409852 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4004410 0x7f83b40048d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:49.928 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83ce11a700 1 -- 172.21.15.3:0/1742409852 >> 172.21.15.3:0/1742409852 conn(0x7f83c80b03a0 msgr2=0x7f83c80b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:49.928 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83ce11a700 1 -- 172.21.15.3:0/1742409852 shutdown_connections 2023-03-03T06:24:49.928 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.916+0000 7f83ce11a700 1 -- 172.21.15.3:0/1742409852 wait complete. 2023-03-03T06:24:49.929 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.920+0000 7f83ce11a700 1 Processor -- start 2023-03-03T06:24:49.929 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.920+0000 7f83ce11a700 1 -- start start 2023-03-03T06:24:49.929 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.920+0000 7f83ce11a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4004410 0x7f83c81542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:49.930 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.920+0000 7f83ce11a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83c81edca0 0x7f83c80c3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:49.930 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.920+0000 7f83ce11a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c80c3b50 0x7f83c815a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:49.930 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.920+0000 7f83ce11a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c815c980 con 0x7f83b4004410 2023-03-03T06:24:49.930 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.920+0000 7f83ce11a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c80c3fc0 con 0x7f83c80c3b50 2023-03-03T06:24:49.930 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.920+0000 7f83ce11a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c81ee3e0 con 0x7f83c81edca0 2023-03-03T06:24:49.931 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83cd118700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4004410 0x7f83c81542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:49.931 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83cd118700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4004410 0x7f83c81542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42776/0 (socket says 172.21.15.3:42776) 2023-03-03T06:24:49.931 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83cd118700 1 -- 172.21.15.3:0/2276777074 learned_addr learned my addr 172.21.15.3:0/2276777074 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:24:49.931 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83cc917700 1 --2- 172.21.15.3:0/2276777074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83c81edca0 0x7f83c80c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:49.932 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83cd919700 1 --2- 172.21.15.3:0/2276777074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c80c3b50 0x7f83c815a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:49.932 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83cd118700 1 -- 172.21.15.3:0/2276777074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83c81edca0 msgr2=0x7f83c80c3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:49.932 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83cd118700 1 --2- 172.21.15.3:0/2276777074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83c81edca0 0x7f83c80c3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:49.932 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83cd118700 1 -- 172.21.15.3:0/2276777074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c80c3b50 msgr2=0x7f83c815a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:49.932 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83cd118700 1 --2- 172.21.15.3:0/2276777074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c80c3b50 0x7f83c815a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:49.932 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83cd118700 1 -- 172.21.15.3:0/2276777074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c80cc6e0 con 0x7f83b4004410 2023-03-03T06:24:49.932 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83cd919700 1 --2- 172.21.15.3:0/2276777074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c80c3b50 0x7f83c815a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:24:49.933 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83cd118700 1 --2- 172.21.15.3:0/2276777074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4004410 0x7f83c81542f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f83c400d820 tx=0x7f83c400db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:49.933 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83bdffb700 1 -- 172.21.15.3:0/2276777074 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4017070 con 0x7f83b4004410 2023-03-03T06:24:49.934 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83bdffb700 1 -- 172.21.15.3:0/2276777074 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c4005300 con 0x7f83b4004410 2023-03-03T06:24:49.934 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83bdffb700 1 -- 172.21.15.3:0/2276777074 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4006880 con 0x7f83b4004410 2023-03-03T06:24:49.934 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83ce11a700 1 -- 172.21.15.3:0/2276777074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c8070f90 con 0x7f83b4004410 2023-03-03T06:24:49.934 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83ce11a700 1 -- 172.21.15.3:0/2276777074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c80cc9c0 con 0x7f83b4004410 2023-03-03T06:24:49.934 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83bdffb700 1 -- 172.21.15.3:0/2276777074 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f83c40069e0 con 0x7f83b4004410 2023-03-03T06:24:49.934 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.924+0000 7f83bdffb700 1 -- 172.21.15.3:0/2276777074 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f83c4006c70 con 0x7f83b4004410 2023-03-03T06:24:49.935 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.928+0000 7f83ce11a700 1 -- 172.21.15.3:0/2276777074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c8072ca0 con 0x7f83b4004410 2023-03-03T06:24:49.938 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:49.932+0000 7f83bdffb700 1 -- 172.21.15.3:0/2276777074 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f83c4006e30 con 0x7f83b4004410 2023-03-03T06:24:50.006 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:50.000+0000 7f83ce11a700 1 -- 172.21.15.3:0/2276777074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83c815c6a0 con 0x7f83b4004410 2023-03-03T06:24:50.006 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:50.000+0000 7f83bdffb700 1 -- 172.21.15.3:0/2276777074 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f83c40070e0 con 0x7f83b4004410 2023-03-03T06:24:50.007 INFO:teuthology.misc.health.smithi003.stdout: 2023-03-03T06:24:50.007 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":10,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:24:48.459752+0000","last_up_change":"0.000000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:24:50.009 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:50.000+0000 7f83ce11a700 1 -- 172.21.15.3:0/2276777074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4004410 msgr2=0x7f83c81542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:50.009 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:50.000+0000 7f83ce11a700 1 --2- 172.21.15.3:0/2276777074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4004410 0x7f83c81542f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f83c400d820 tx=0x7f83c400db30 comp rx=0 tx=0).stop 2023-03-03T06:24:50.010 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:50.004+0000 7f83ce11a700 1 -- 172.21.15.3:0/2276777074 shutdown_connections 2023-03-03T06:24:50.010 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:50.004+0000 7f83ce11a700 1 --2- 172.21.15.3:0/2276777074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83c80c3b50 0x7f83c815a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:50.010 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:50.004+0000 7f83ce11a700 1 --2- 172.21.15.3:0/2276777074 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83c81edca0 0x7f83c80c3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:50.010 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:50.004+0000 7f83ce11a700 1 --2- 172.21.15.3:0/2276777074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83b4004410 0x7f83c81542f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:50.011 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:50.004+0000 7f83ce11a700 1 -- 172.21.15.3:0/2276777074 >> 172.21.15.3:0/2276777074 conn(0x7f83c80b03a0 msgr2=0x7f83c8159340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:50.011 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:50.004+0000 7f83ce11a700 1 -- 172.21.15.3:0/2276777074 shutdown_connections 2023-03-03T06:24:50.011 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:50.004+0000 7f83ce11a700 1 -- 172.21.15.3:0/2276777074 wait complete. 2023-03-03T06:24:50.033 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-03-03T06:24:50.300 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:50.294+0000 7f76c27c12c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-03T06:24:50.428 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:50.422+0000 7f76c27c12c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-03T06:24:50.445 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:50.436+0000 7fd2c67da2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-03T06:24:50.579 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:50.572+0000 7fd2c67da2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-03T06:24:50.645 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:50.638+0000 7f76c27c12c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-03T06:24:50.771 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:50.762+0000 7f76c27c12c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-03T06:24:50.792 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:50.784+0000 7fd2c67da2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-03T06:24:50.918 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:50.912+0000 7fd2c67da2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-03T06:24:51.086 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:51.078+0000 7f76c27c12c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-03T06:24:51.236 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:51.228+0000 7fd2c67da2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-03T06:24:51.355 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:51.346+0000 7f76c27c12c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-03T06:24:51.491 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:51.482+0000 7f76c27c12c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-03T06:24:51.510 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:51.504+0000 7fd2c67da2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-03T06:24:51.648 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:51.640+0000 7fd2c67da2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-03T06:24:52.361 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:52.354+0000 7f76c27c12c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-03T06:24:52.606 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:24:52.600+0000 7fd2c67da2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-03T06:24:53.408 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:53.402+0000 7f76a8ff5700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-03-03T06:24:53.408 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:24:53.402+0000 7f76a8ff5700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-03-03T06:24:53.501 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-03-03T06:24:53.492+0000 7f3a305ec700 -1 osd.2 0 log_to_monitors true 2023-03-03T06:24:53.523 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-03-03T06:24:53.520+0000 7f5f1307c700 -1 osd.1 0 log_to_monitors true 2023-03-03T06:24:53.533 INFO:tasks.ceph.osd.6.smithi032.stderr:2023-03-03T06:24:53.526+0000 7f0420954700 -1 osd.6 0 log_to_monitors true 2023-03-03T06:24:53.542 INFO:tasks.ceph.osd.4.smithi032.stderr:2023-03-03T06:24:53.534+0000 7ff939816700 -1 osd.4 0 log_to_monitors true 2023-03-03T06:24:53.584 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-03-03T06:24:53.576+0000 7fbe73cd7700 -1 osd.3 0 log_to_monitors true 2023-03-03T06:24:53.585 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-03-03T06:24:53.576+0000 7f69f47c4700 -1 osd.0 0 log_to_monitors true 2023-03-03T06:24:53.621 INFO:tasks.ceph.osd.7.smithi032.stderr:2023-03-03T06:24:53.614+0000 7f03d9388700 -1 osd.7 0 log_to_monitors true 2023-03-03T06:24:53.642 INFO:tasks.ceph.osd.5.smithi032.stderr:2023-03-03T06:24:53.634+0000 7f0058d71700 -1 osd.5 0 log_to_monitors true 2023-03-03T06:24:56.840 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-03T06:24:57.070 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.064+0000 7f65e1fe8700 1 Processor -- start 2023-03-03T06:24:57.070 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.064+0000 7f65e1fe8700 1 -- start start 2023-03-03T06:24:57.072 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.064+0000 7f65e1fe8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc0b8870 0x7f65dc0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:57.073 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.064+0000 7f65e1fe8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65dc0d32a0 0x7f65dc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:57.073 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.064+0000 7f65e1fe8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65dc0cd370 con 0x7f65dc0d3710 2023-03-03T06:24:57.073 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.064+0000 7f65e1fe8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65dc0cd610 con 0x7f65dc0d32a0 2023-03-03T06:24:57.074 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.064+0000 7f65e1fe8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc0d07f0 con 0x7f65dc0b8870 2023-03-03T06:24:57.074 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65e17e7700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f65dc0d3710 0x7f65dc0d0ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:44262/0 (socket says 172.21.15.3:44262) 2023-03-03T06:24:57.075 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65e17e7700 1 -- 172.21.15.3:0/3494012919 learned_addr learned my addr 172.21.15.3:0/3494012919 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:24:57.075 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65e0fe6700 1 --2- 172.21.15.3:0/3494012919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc0b8870 0x7f65dc0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:57.075 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65dbfff700 1 --2- 172.21.15.3:0/3494012919 >> v2:172.21.15.3:3300/0 conn(0x7f65dc0d32a0 0x7f65dc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:57.075 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65db7fe700 1 -- 172.21.15.3:0/3494012919 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703181681 0 0) 0x7f65dc0cd370 con 0x7f65dc0d3710 2023-03-03T06:24:57.076 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65db7fe700 1 -- 172.21.15.3:0/3494012919 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65cc0035d0 con 0x7f65dc0d3710 2023-03-03T06:24:57.076 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65dbfff700 1 -- 172.21.15.3:0/3494012919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc0b8870 msgr2=0x7f65dc0b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:24:57.076 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65dbfff700 1 --2- 172.21.15.3:0/3494012919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc0b8870 0x7f65dc0b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:57.076 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65dbfff700 1 -- 172.21.15.3:0/3494012919 >> v1:172.21.15.32:6789/0 conn(0x7f65dc0d3710 legacy=0x7f65dc0d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:57.076 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65dbfff700 1 -- 172.21.15.3:0/3494012919 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc0d0ad0 con 0x7f65dc0d32a0 2023-03-03T06:24:57.077 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65dbfff700 1 --2- 172.21.15.3:0/3494012919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc0d32a0 0x7f65dc0cf590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009850 tx=0x7f65c8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0214c3553cd700 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:57.077 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65db7fe700 1 -- 172.21.15.3:0/3494012919 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8011830 con 0x7f65dc0d32a0 2023-03-03T06:24:57.077 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65db7fe700 1 -- 172.21.15.3:0/3494012919 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c8011990 con 0x7f65dc0d32a0 2023-03-03T06:24:57.077 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65db7fe700 1 -- 172.21.15.3:0/3494012919 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8011d40 con 0x7f65dc0d32a0 2023-03-03T06:24:57.077 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65e1fe8700 1 -- 172.21.15.3:0/3494012919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc0d32a0 msgr2=0x7f65dc0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:57.078 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.068+0000 7f65e1fe8700 1 --2- 172.21.15.3:0/3494012919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc0d32a0 0x7f65dc0cf590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009850 tx=0x7f65c8009b60 comp rx=0 tx=0).stop 2023-03-03T06:24:57.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:24:57.079 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.072+0000 7f65e1fe8700 1 -- 172.21.15.3:0/3494012919 shutdown_connections 2023-03-03T06:24:57.079 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.072+0000 7f65e1fe8700 1 --2- 172.21.15.3:0/3494012919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc0d32a0 0x7f65dc0cf590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:57.079 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.072+0000 7f65e1fe8700 1 --2- 172.21.15.3:0/3494012919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc0b8870 0x7f65dc0b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:57.079 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.072+0000 7f65e1fe8700 1 -- 172.21.15.3:0/3494012919 >> 172.21.15.3:0/3494012919 conn(0x7f65dc0714a0 msgr2=0x7f65dc071870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:57.079 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.072+0000 7f65e1fe8700 1 -- 172.21.15.3:0/3494012919 shutdown_connections 2023-03-03T06:24:57.080 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.072+0000 7f65e1fe8700 1 -- 172.21.15.3:0/3494012919 wait complete. 2023-03-03T06:24:57.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:24:57.087 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.076+0000 7f65e1fe8700 1 Processor -- start 2023-03-03T06:24:57.088 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.076+0000 7f65e1fe8700 1 -- start start 2023-03-03T06:24:57.088 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.076+0000 7f65e1fe8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc0b8870 0x7f65dc154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:57.088 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.076+0000 7f65e1fe8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc0d32a0 0x7f65dc0cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:57.088 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.076+0000 7f65dbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc0d32a0 0x7f65dc0cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:57.089 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65dbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc0d32a0 0x7f65dc0cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41480/0 (socket says 172.21.15.3:41480) 2023-03-03T06:24:57.089 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65dbfff700 1 -- 172.21.15.3:0/842204967 learned_addr learned my addr 172.21.15.3:0/842204967 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:24:57.089 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65e1fe8700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc0d3710 0x7f65dc1f9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:57.089 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65e0fe6700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc0b8870 0x7f65dc154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:57.089 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65e17e7700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc0d3710 0x7f65dc1f9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:57.090 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65e1fe8700 1 -- 172.21.15.3:0/842204967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc155c40 con 0x7f65dc0d32a0 2023-03-03T06:24:57.090 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65e1fe8700 1 -- 172.21.15.3:0/842204967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc155680 con 0x7f65dc0d3710 2023-03-03T06:24:57.090 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65e1fe8700 1 -- 172.21.15.3:0/842204967 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc155960 con 0x7f65dc0b8870 2023-03-03T06:24:57.090 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65dbfff700 1 -- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc0b8870 msgr2=0x7f65dc154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:57.090 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65dbfff700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc0b8870 0x7f65dc154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:57.091 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65dbfff700 1 -- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc0d3710 msgr2=0x7f65dc1f9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:57.091 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65dbfff700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc0d3710 0x7f65dc1f9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:57.091 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65dbfff700 1 -- 172.21.15.3:0/842204967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80094b0 con 0x7f65dc0d32a0 2023-03-03T06:24:57.092 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65dbfff700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc0d32a0 0x7f65dc0cc710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0d3c80 tx=0x7f65c8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:57.093 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65d97fa700 1 -- 172.21.15.3:0/842204967 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8004480 con 0x7f65dc0d32a0 2023-03-03T06:24:57.093 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65d97fa700 1 -- 172.21.15.3:0/842204967 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c801f070 con 0x7f65dc0d32a0 2023-03-03T06:24:57.093 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65d97fa700 1 -- 172.21.15.3:0/842204967 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80038e0 con 0x7f65dc0d32a0 2023-03-03T06:24:57.093 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65e1fe8700 1 -- 172.21.15.3:0/842204967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65dc0cf300 con 0x7f65dc0d32a0 2023-03-03T06:24:57.094 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.080+0000 7f65e1fe8700 1 -- 172.21.15.3:0/842204967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc0cf170 con 0x7f65dc0d32a0 2023-03-03T06:24:57.094 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.084+0000 7f65c6ffd700 1 -- 172.21.15.3:0/842204967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b4005310 con 0x7f65dc0d32a0 2023-03-03T06:24:57.094 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.088+0000 7f65d97fa700 1 -- 172.21.15.3:0/842204967 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 83919+0+0 (secure 0 0 0) 0x7f65c802a020 con 0x7f65dc0d32a0 2023-03-03T06:24:57.095 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.088+0000 7f65d97fa700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f65c0064e90 0x7f65c0067340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:24:57.096 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.088+0000 7f65d97fa700 1 -- 172.21.15.3:0/842204967 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f65c8082690 con 0x7f65dc0d32a0 2023-03-03T06:24:57.097 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.088+0000 7f65e0fe6700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f65c0064e90 0x7f65c0067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:24:57.098 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.092+0000 7f65e0fe6700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f65c0064e90 0x7f65c0067340 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009950 tx=0x7f65d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:24:57.109 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.100+0000 7f65d97fa700 1 -- 172.21.15.3:0/842204967 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f65c8051680 con 0x7f65dc0d32a0 2023-03-03T06:24:57.469 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.460+0000 7f65c6ffd700 1 -- 172.21.15.3:0/842204967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65b40055c0 con 0x7f65dc0d32a0 2023-03-03T06:24:57.470 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.464+0000 7f65d97fa700 1 -- 172.21.15.3:0/842204967 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f65c8006ce0 con 0x7f65dc0d32a0 2023-03-03T06:24:57.476 INFO:teuthology.misc.health.smithi003.stdout: 2023-03-03T06:24:57.476 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":12,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:24:55.427991+0000","last_up_change":"0.000000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:24:57.485 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.476+0000 7f65c6ffd700 1 -- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f65c0064e90 msgr2=0x7f65c0067340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:57.485 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.476+0000 7f65c6ffd700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f65c0064e90 0x7f65c0067340 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009950 tx=0x7f65d0008040 comp rx=0 tx=0).stop 2023-03-03T06:24:57.485 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.476+0000 7f65c6ffd700 1 -- 172.21.15.3:0/842204967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc0d32a0 msgr2=0x7f65dc0cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:24:57.485 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.476+0000 7f65c6ffd700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc0d32a0 0x7f65dc0cc710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0d3c80 tx=0x7f65c8004770 comp rx=0 tx=0).stop 2023-03-03T06:24:57.486 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.480+0000 7f65c6ffd700 1 -- 172.21.15.3:0/842204967 shutdown_connections 2023-03-03T06:24:57.486 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.480+0000 7f65c6ffd700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f65c0064e90 0x7f65c0067340 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:57.486 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.480+0000 7f65c6ffd700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65dc0d3710 0x7f65dc1f9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:57.487 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.480+0000 7f65c6ffd700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65dc0d32a0 0x7f65dc0cc710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:57.487 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.480+0000 7f65c6ffd700 1 --2- 172.21.15.3:0/842204967 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f65dc0b8870 0x7f65dc154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:24:57.487 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.480+0000 7f65c6ffd700 1 -- 172.21.15.3:0/842204967 >> 172.21.15.3:0/842204967 conn(0x7f65dc0714a0 msgr2=0x7f65dc071870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:24:57.487 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.480+0000 7f65c6ffd700 1 -- 172.21.15.3:0/842204967 shutdown_connections 2023-03-03T06:24:57.487 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:24:57.480+0000 7f65c6ffd700 1 -- 172.21.15.3:0/842204967 wait complete. 2023-03-03T06:24:57.528 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-03-03T06:24:57.656 INFO:tasks.ceph.osd.6.smithi032.stderr:2023-03-03T06:24:57.646+0000 7f041c793700 -1 osd.6 0 waiting for initial osdmap 2023-03-03T06:24:57.668 INFO:tasks.ceph.osd.6.smithi032.stderr:2023-03-03T06:24:57.662+0000 7f04141f0700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-03T06:24:57.741 INFO:tasks.ceph.osd.5.smithi032.stderr:2023-03-03T06:24:57.734+0000 7f0054bb0700 -1 osd.5 0 waiting for initial osdmap 2023-03-03T06:24:57.749 INFO:tasks.ceph.osd.5.smithi032.stderr:2023-03-03T06:24:57.742+0000 7f004c5f0700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-03T06:24:57.759 INFO:tasks.ceph.osd.4.smithi032.stderr:2023-03-03T06:24:57.754+0000 7ff935fdc700 -1 osd.4 0 waiting for initial osdmap 2023-03-03T06:24:57.768 INFO:tasks.ceph.osd.4.smithi032.stderr:2023-03-03T06:24:57.762+0000 7ff92cff0700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-03T06:24:57.779 INFO:tasks.ceph.osd.7.smithi032.stderr:2023-03-03T06:24:57.774+0000 7f03d51c7700 -1 osd.7 0 waiting for initial osdmap 2023-03-03T06:24:57.793 INFO:tasks.ceph.osd.7.smithi032.stderr:2023-03-03T06:24:57.786+0000 7f03ccbf0700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-03T06:24:58.254 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-03-03T06:24:58.244+0000 7f3a2c42b700 -1 osd.2 0 waiting for initial osdmap 2023-03-03T06:24:58.265 INFO:tasks.ceph.osd.2.smithi003.stderr:2023-03-03T06:24:58.256+0000 7f3a23df0700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-03T06:24:58.307 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-03-03T06:24:58.300+0000 7f69f0f8a700 -1 osd.0 0 waiting for initial osdmap 2023-03-03T06:24:58.319 INFO:tasks.ceph.osd.0.smithi003.stderr:2023-03-03T06:24:58.312+0000 7f69e7ff0700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-03T06:24:58.323 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-03-03T06:24:58.316+0000 7f5f0f842700 -1 osd.1 0 waiting for initial osdmap 2023-03-03T06:24:58.334 INFO:tasks.ceph.osd.1.smithi003.stderr:2023-03-03T06:24:58.328+0000 7f5f067f0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-03T06:24:58.369 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-03-03T06:24:58.364+0000 7fbe6fb16700 -1 osd.3 0 waiting for initial osdmap 2023-03-03T06:24:58.383 INFO:tasks.ceph.osd.3.smithi003.stderr:2023-03-03T06:24:58.376+0000 7fbe673f0700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-03T06:25:04.335 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-03T06:25:04.520 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.516+0000 7fd486662700 1 Processor -- start 2023-03-03T06:25:04.520 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.516+0000 7fd486662700 1 -- start start 2023-03-03T06:25:04.522 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd486662700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4800b5fc0 0x7fd4800b6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:04.523 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd486662700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4800b6950 con 0x7fd48015c480 2023-03-03T06:25:04.523 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd486662700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4800d2db0 con 0x7fd4800d2080 2023-03-03T06:25:04.523 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd486662700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48006a480 con 0x7fd4800b5fc0 2023-03-03T06:25:04.523 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd48015c480 0x7fd480160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51224/0 (socket says 172.21.15.3:51224) 2023-03-03T06:25:04.523 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47ffff700 1 -- 172.21.15.3:0/515871440 learned_addr learned my addr 172.21.15.3:0/515871440 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:04.525 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd485400700 1 --2- 172.21.15.3:0/515871440 >> v2:172.21.15.32:3300/0 conn(0x7fd4800b5fc0 0x7fd4800b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:04.525 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725837371 0 0) 0x7fd470001490 con 0x7fd48015c480 2023-03-03T06:25:04.525 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd46c0035d0 con 0x7fd48015c480 2023-03-03T06:25:04.525 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 581331823 0 0) 0x7fd46c0035d0 con 0x7fd48015c480 2023-03-03T06:25:04.525 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 >> v2:172.21.15.32:3300/0 conn(0x7fd4800b5fc0 msgr2=0x7fd4800b6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:04.526 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 --2- 172.21.15.3:0/515871440 >> v2:172.21.15.32:3300/0 conn(0x7fd4800b5fc0 0x7fd4800b6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.526 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 >> v1:172.21.15.32:6789/0 conn(0x7fd4800d2080 legacy=0x7fd480069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:04.526 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4800d2510 con 0x7fd48015c480 2023-03-03T06:25:04.526 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7fd470001b80 con 0x7fd48015c480 2023-03-03T06:25:04.526 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 >> v1:172.21.15.3:6789/0 conn(0x7fd48015c480 legacy=0x7fd480160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:04.526 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 --2- 172.21.15.3:0/515871440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd46c003a90 0x7fd46c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:04.526 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 --2- 172.21.15.3:0/515871440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c004870 0x7fd46c004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:04.527 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 --2- 172.21.15.3:0/515871440 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd46c005270 0x7fd46c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:04.527 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48006a480 con 0x7fd46c004870 2023-03-03T06:25:04.527 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c00c1b0 con 0x7fd46c003a90 2023-03-03T06:25:04.527 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c00c370 con 0x7fd46c005270 2023-03-03T06:25:04.527 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd4777fe700 1 --2- 172.21.15.3:0/515871440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c004870 0x7fd46c004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:04.527 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.520+0000 7fd47ffff700 1 --2- 172.21.15.3:0/515871440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd46c003a90 0x7fd46c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:04.529 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd485400700 1 --2- 172.21.15.3:0/515871440 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd46c005270 0x7fd46c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:04.529 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd4777fe700 1 -- 172.21.15.3:0/515871440 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd46c005270 msgr2=0x7fd46c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:04.529 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd4777fe700 1 --2- 172.21.15.3:0/515871440 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd46c005270 0x7fd46c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.529 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd4777fe700 1 -- 172.21.15.3:0/515871440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd46c003a90 msgr2=0x7fd46c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:04.530 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd4777fe700 1 --2- 172.21.15.3:0/515871440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd46c003a90 0x7fd46c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.530 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd4777fe700 1 -- 172.21.15.3:0/515871440 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c0035d0 con 0x7fd46c004870 2023-03-03T06:25:04.530 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd485400700 1 -- 172.21.15.3:0/515871440 reap_dead start 2023-03-03T06:25:04.530 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd4777fe700 1 --2- 172.21.15.3:0/515871440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c004870 0x7fd46c004ce0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd468007fd0 tx=0x7fd46c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab22c06a40b1bb8 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:04.530 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680052d0 con 0x7fd46c004870 2023-03-03T06:25:04.530 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd468005430 con 0x7fd46c004870 2023-03-03T06:25:04.530 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd47f7fe700 1 -- 172.21.15.3:0/515871440 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680055a0 con 0x7fd46c004870 2023-03-03T06:25:04.531 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd486662700 1 -- 172.21.15.3:0/515871440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c004870 msgr2=0x7fd46c004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:04.531 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd486662700 1 --2- 172.21.15.3:0/515871440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c004870 0x7fd46c004ce0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd468007fd0 tx=0x7fd46c0058a0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.531 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd486662700 1 -- 172.21.15.3:0/515871440 shutdown_connections 2023-03-03T06:25:04.531 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd486662700 1 --2- 172.21.15.3:0/515871440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c004870 0x7fd46c004ce0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.531 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd486662700 1 -- 172.21.15.3:0/515871440 >> 172.21.15.3:0/515871440 conn(0x7fd4800d0fc0 msgr2=0x7fd4800d13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:04.531 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd486662700 1 -- 172.21.15.3:0/515871440 shutdown_connections 2023-03-03T06:25:04.531 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.524+0000 7fd486662700 1 -- 172.21.15.3:0/515871440 wait complete. 2023-03-03T06:25:04.532 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.528+0000 7fd486662700 1 Processor -- start 2023-03-03T06:25:04.532 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.528+0000 7fd486662700 1 -- start start 2023-03-03T06:25:04.534 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd486662700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c004870 0x7fd4801539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:04.534 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd486662700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd480154540 0x7fd48006b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:04.534 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd486662700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48006ba80 0x7fd4800c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:04.534 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd486662700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48006bef0 con 0x7fd46c004870 2023-03-03T06:25:04.534 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd486662700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd480153f10 con 0x7fd48006ba80 2023-03-03T06:25:04.535 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd486662700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd480156e20 con 0x7fd480154540 2023-03-03T06:25:04.535 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd47ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c004870 0x7fd4801539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:04.536 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd47ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c004870 0x7fd4801539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53692/0 (socket says 172.21.15.3:53692) 2023-03-03T06:25:04.536 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd47ffff700 1 -- 172.21.15.3:0/2884453867 learned_addr learned my addr 172.21.15.3:0/2884453867 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:04.536 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd4777fe700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd480154540 0x7fd48006b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:04.536 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd485400700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48006ba80 0x7fd4800c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:04.536 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd47ffff700 1 -- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd480154540 msgr2=0x7fd48006b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:04.536 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd47ffff700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd480154540 0x7fd48006b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.537 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd47ffff700 1 -- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48006ba80 msgr2=0x7fd4800c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:04.537 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd47ffff700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48006ba80 0x7fd4800c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.537 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd47ffff700 1 -- 172.21.15.3:0/2884453867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4800d2ad0 con 0x7fd46c004870 2023-03-03T06:25:04.538 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd47ffff700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c004870 0x7fd4801539d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd470001ba0 tx=0x7fd470001bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:04.538 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd47d7fa700 1 -- 172.21.15.3:0/2884453867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4700077c0 con 0x7fd46c004870 2023-03-03T06:25:04.538 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd47d7fa700 1 -- 172.21.15.3:0/2884453867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd470007920 con 0x7fd46c004870 2023-03-03T06:25:04.538 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd486662700 1 -- 172.21.15.3:0/2884453867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4800d27f0 con 0x7fd46c004870 2023-03-03T06:25:04.538 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd47d7fa700 1 -- 172.21.15.3:0/2884453867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4700133f0 con 0x7fd46c004870 2023-03-03T06:25:04.538 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.532+0000 7fd486662700 1 -- 172.21.15.3:0/2884453867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4800cf130 con 0x7fd46c004870 2023-03-03T06:25:04.539 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.536+0000 7fd486662700 1 -- 172.21.15.3:0/2884453867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4800ba4c0 con 0x7fd46c004870 2023-03-03T06:25:04.540 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.536+0000 7fd47d7fa700 1 -- 172.21.15.3:0/2884453867 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) v1 ==== 83919+0+0 (secure 0 0 0) 0x7fd470023070 con 0x7fd46c004870 2023-03-03T06:25:04.543 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.540+0000 7fd47d7fa700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd460064e90 0x7fd460067340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:04.543 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.540+0000 7fd47d7fa700 1 -- 172.21.15.3:0/2884453867 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd4700821d0 con 0x7fd46c004870 2023-03-03T06:25:04.543 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.540+0000 7fd4777fe700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd460064e90 0x7fd460067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:04.545 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.544+0000 7fd4777fe700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd460064e90 0x7fd460067340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd468015040 tx=0x7fd468009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:04.550 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.548+0000 7fd47d7fa700 1 -- 172.21.15.3:0/2884453867 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd4700506a0 con 0x7fd46c004870 2023-03-03T06:25:04.736 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.732+0000 7fd486662700 1 -- 172.21.15.3:0/2884453867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4800b6950 con 0x7fd46c004870 2023-03-03T06:25:04.738 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.736+0000 7fd47d7fa700 1 -- 172.21.15.3:0/2884453867 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12286 (secure 0 0 0) 0x7fd4800b6950 con 0x7fd46c004870 2023-03-03T06:25:04.739 INFO:teuthology.misc.health.smithi003.stdout: 2023-03-03T06:25:04.739 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":17,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:02.704484+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:25:04.741 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.740+0000 7fd486662700 1 -- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd460064e90 msgr2=0x7fd460067340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:04.742 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.740+0000 7fd486662700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd460064e90 0x7fd460067340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd468015040 tx=0x7fd468009030 comp rx=0 tx=0).stop 2023-03-03T06:25:04.742 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.740+0000 7fd486662700 1 -- 172.21.15.3:0/2884453867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c004870 msgr2=0x7fd4801539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:04.742 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.740+0000 7fd486662700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c004870 0x7fd4801539d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd470001ba0 tx=0x7fd470001bd0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.743 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.740+0000 7fd486662700 1 -- 172.21.15.3:0/2884453867 shutdown_connections 2023-03-03T06:25:04.743 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.740+0000 7fd486662700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd460064e90 0x7fd460067340 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.744 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.740+0000 7fd486662700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd48006ba80 0x7fd4800c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.744 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.740+0000 7fd486662700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd480154540 0x7fd48006b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.744 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.740+0000 7fd486662700 1 --2- 172.21.15.3:0/2884453867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd46c004870 0x7fd4801539d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.744 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.740+0000 7fd486662700 1 -- 172.21.15.3:0/2884453867 >> 172.21.15.3:0/2884453867 conn(0x7fd4800d0fc0 msgr2=0x7fd4801592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:04.744 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.740+0000 7fd486662700 1 -- 172.21.15.3:0/2884453867 shutdown_connections 2023-03-03T06:25:04.744 INFO:teuthology.misc.health.smithi003.stderr:2023-03-03T06:25:04.740+0000 7fd486662700 1 -- 172.21.15.3:0/2884453867 wait complete. 2023-03-03T06:25:04.771 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-03-03T06:25:04.772 INFO:tasks.ceph:Creating RBD pool 2023-03-03T06:25:04.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd pool create rbd 8 2023-03-03T06:25:04.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.964+0000 7ff62229f700 1 Processor -- start 2023-03-03T06:25:04.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.964+0000 7ff62229f700 1 -- start start 2023-03-03T06:25:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.968+0000 7ff62229f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff61c15d6e0 0x7ff61c0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.968+0000 7ff62229f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c0cf3b0 con 0x7ff61c0cfd80 2023-03-03T06:25:04.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.968+0000 7ff62229f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c156f70 con 0x7ff61c156ba0 2023-03-03T06:25:04.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.968+0000 7ff62229f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c0cd320 con 0x7ff61c15d6e0 2023-03-03T06:25:04.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.968+0000 7ff62129d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff61c156ba0 0x7ff61c071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:39512/0 (socket says 172.21.15.3:39512) 2023-03-03T06:25:04.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.968+0000 7ff62129d700 1 -- 172.21.15.3:0/477906094 learned_addr learned my addr 172.21.15.3:0/477906094 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:04.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.968+0000 7ff620a9c700 1 --2- 172.21.15.3:0/477906094 >> v2:172.21.15.32:3300/0 conn(0x7ff61c15d6e0 0x7ff61c0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:04.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.968+0000 7ff620a9c700 1 -- 172.21.15.3:0/477906094 >> v1:172.21.15.32:6789/0 conn(0x7ff61c156ba0 legacy=0x7ff61c071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:04.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.968+0000 7ff620a9c700 1 -- 172.21.15.3:0/477906094 >> v1:172.21.15.3:6789/0 conn(0x7ff61c0cfd80 legacy=0x7ff61c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:04.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.968+0000 7ff620a9c700 1 -- 172.21.15.3:0/477906094 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c0cd600 con 0x7ff61c15d6e0 2023-03-03T06:25:04.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.968+0000 7ff613fff700 1 -- 172.21.15.3:0/477906094 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455245327 0 0) 0x7ff61c0cf3b0 con 0x7ff61c0cfd80 2023-03-03T06:25:04.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.972+0000 7ff620a9c700 1 --2- 172.21.15.3:0/477906094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c15d6e0 0x7ff61c0cf840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009850 tx=0x7ff60c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff615811a238834c server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:04.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.972+0000 7ff613fff700 1 -- 172.21.15.3:0/477906094 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c011830 con 0x7ff61c15d6e0 2023-03-03T06:25:04.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.972+0000 7ff613fff700 1 -- 172.21.15.3:0/477906094 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c011990 con 0x7ff61c15d6e0 2023-03-03T06:25:04.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.972+0000 7ff613fff700 1 -- 172.21.15.3:0/477906094 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c011d20 con 0x7ff61c15d6e0 2023-03-03T06:25:04.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.972+0000 7ff62229f700 1 -- 172.21.15.3:0/477906094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c15d6e0 msgr2=0x7ff61c0cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:04.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.972+0000 7ff62229f700 1 --2- 172.21.15.3:0/477906094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c15d6e0 0x7ff61c0cf840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009850 tx=0x7ff60c009b60 comp rx=0 tx=0).stop 2023-03-03T06:25:04.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.972+0000 7ff62229f700 1 -- 172.21.15.3:0/477906094 shutdown_connections 2023-03-03T06:25:04.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.972+0000 7ff62229f700 1 --2- 172.21.15.3:0/477906094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c15d6e0 0x7ff61c0cf840 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.972+0000 7ff62229f700 1 -- 172.21.15.3:0/477906094 >> 172.21.15.3:0/477906094 conn(0x7ff61c06e5c0 msgr2=0x7ff61c06e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:04.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.972+0000 7ff62229f700 1 -- 172.21.15.3:0/477906094 shutdown_connections 2023-03-03T06:25:04.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.972+0000 7ff62229f700 1 -- 172.21.15.3:0/477906094 wait complete. 2023-03-03T06:25:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.976+0000 7ff62229f700 1 Processor -- start 2023-03-03T06:25:04.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.976+0000 7ff62229f700 1 -- start start 2023-03-03T06:25:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.976+0000 7ff62229f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c0cfd80 0x7ff61c1596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.976+0000 7ff62229f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff61c156ba0 0x7ff61c1f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.976+0000 7ff62229f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff61c15d6e0 0x7ff61c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.976+0000 7ff62229f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0cb820 con 0x7ff61c156ba0 2023-03-03T06:25:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.976+0000 7ff62229f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0caef0 con 0x7ff61c0cfd80 2023-03-03T06:25:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.976+0000 7ff62229f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0cb1d0 con 0x7ff61c15d6e0 2023-03-03T06:25:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.976+0000 7ff620a9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff61c156ba0 0x7ff61c1f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff620a9c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff61c156ba0 0x7ff61c1f2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53700/0 (socket says 172.21.15.3:53700) 2023-03-03T06:25:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff620a9c700 1 -- 172.21.15.3:0/1341380983 learned_addr learned my addr 172.21.15.3:0/1341380983 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff62129d700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c0cfd80 0x7ff61c1596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff621a9e700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff61c15d6e0 0x7ff61c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff620a9c700 1 -- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff61c15d6e0 msgr2=0x7ff61c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff620a9c700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff61c15d6e0 0x7ff61c0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff620a9c700 1 -- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c0cfd80 msgr2=0x7ff61c1596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff620a9c700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c0cfd80 0x7ff61c1596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff620a9c700 1 -- 172.21.15.3:0/1341380983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c0094b0 con 0x7ff61c156ba0 2023-03-03T06:25:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff620a9c700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff61c156ba0 0x7ff61c1f2f90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff60c014040 tx=0x7ff60c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff611ffb700 1 -- 172.21.15.3:0/1341380983 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c00ba70 con 0x7ff61c156ba0 2023-03-03T06:25:04.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff611ffb700 1 -- 172.21.15.3:0/1341380983 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c01e070 con 0x7ff61c156ba0 2023-03-03T06:25:04.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff611ffb700 1 -- 172.21.15.3:0/1341380983 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c00bdf0 con 0x7ff61c156ba0 2023-03-03T06:25:04.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff62229f700 1 -- 172.21.15.3:0/1341380983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c0cef40 con 0x7ff61c156ba0 2023-03-03T06:25:04.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.980+0000 7ff62229f700 1 -- 172.21.15.3:0/1341380983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c0cd8e0 con 0x7ff61c156ba0 2023-03-03T06:25:04.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.984+0000 7ff62229f700 1 -- 172.21.15.3:0/1341380983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c0cf0d0 con 0x7ff61c156ba0 2023-03-03T06:25:04.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.984+0000 7ff611ffb700 1 -- 172.21.15.3:0/1341380983 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff60c02a020 con 0x7ff61c156ba0 2023-03-03T06:25:04.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.988+0000 7ff611ffb700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff600064ee0 0x7ff600067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:04.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.988+0000 7ff611ffb700 1 -- 172.21.15.3:0/1341380983 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff60c082db0 con 0x7ff61c156ba0 2023-03-03T06:25:04.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.988+0000 7ff62129d700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff600064ee0 0x7ff600067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:04.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.988+0000 7ff62129d700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff600064ee0 0x7ff600067390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0b32e0 tx=0x7ff618007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:05.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:04.996+0000 7ff611ffb700 1 -- 172.21.15.3:0/1341380983 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff60c087070 con 0x7ff61c156ba0 2023-03-03T06:25:05.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:05.180+0000 7ff62229f700 1 -- 172.21.15.3:0/1341380983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7ff61c073e90 con 0x7ff61c156ba0 2023-03-03T06:25:06.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.180+0000 7ff611ffb700 1 -- 172.21.15.3:0/1341380983 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v18) v1 ==== 109+0+0 (secure 0 0 0) 0x7ff60c051310 con 0x7ff61c156ba0 2023-03-03T06:25:06.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.184+0000 7ff62229f700 1 -- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff600064ee0 msgr2=0x7ff600067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:06.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.184+0000 7ff62229f700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff600064ee0 0x7ff600067390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0b32e0 tx=0x7ff618007f30 comp rx=0 tx=0).stop 2023-03-03T06:25:06.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.184+0000 7ff62229f700 1 -- 172.21.15.3:0/1341380983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff61c156ba0 msgr2=0x7ff61c1f2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:06.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.184+0000 7ff62229f700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff61c156ba0 0x7ff61c1f2f90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff60c014040 tx=0x7ff60c004770 comp rx=0 tx=0).stop 2023-03-03T06:25:06.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.184+0000 7ff62229f700 1 -- 172.21.15.3:0/1341380983 shutdown_connections 2023-03-03T06:25:06.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.184+0000 7ff62229f700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff600064ee0 0x7ff600067390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:06.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.184+0000 7ff62229f700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff61c15d6e0 0x7ff61c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:06.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.184+0000 7ff62229f700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff61c156ba0 0x7ff61c1f2f90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:06.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.184+0000 7ff62229f700 1 --2- 172.21.15.3:0/1341380983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c0cfd80 0x7ff61c1596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:06.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.184+0000 7ff62229f700 1 -- 172.21.15.3:0/1341380983 >> 172.21.15.3:0/1341380983 conn(0x7ff61c06e5c0 msgr2=0x7ff61c15f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:06.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.184+0000 7ff62229f700 1 -- 172.21.15.3:0/1341380983 shutdown_connections 2023-03-03T06:25:06.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.184+0000 7ff62229f700 1 -- 172.21.15.3:0/1341380983 wait complete. 2023-03-03T06:25:06.193 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'rbd' created 2023-03-03T06:25:06.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-03-03T06:25:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.416+0000 7faed7466700 1 Processor -- start 2023-03-03T06:25:06.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.416+0000 7faed7466700 1 -- start start 2023-03-03T06:25:06.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.416+0000 7faed7466700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faed0159240 0x7faed00cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:06.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.416+0000 7faed7466700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed0070020 con 0x7faed00ce150 2023-03-03T06:25:06.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.416+0000 7faed7466700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed0070300 con 0x7faed00cdc50 2023-03-03T06:25:06.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.416+0000 7faed7466700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed015dd40 con 0x7faed0159240 2023-03-03T06:25:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed6c65700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7faed00ce150 0x7faed00ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51244/0 (socket says 172.21.15.3:51244) 2023-03-03T06:25:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed6c65700 1 -- 172.21.15.3:0/1323209901 learned_addr learned my addr 172.21.15.3:0/1323209901 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed6464700 1 --2- 172.21.15.3:0/1323209901 >> v2:172.21.15.32:3300/0 conn(0x7faed0159240 0x7faed00cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645930201 0 0) 0x7faed0070020 con 0x7faed00ce150 2023-03-03T06:25:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faebc0057e0 con 0x7faed00ce150 2023-03-03T06:25:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 748345934 0 0) 0x7faebc0057e0 con 0x7faed00ce150 2023-03-03T06:25:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 >> v2:172.21.15.32:3300/0 conn(0x7faed0159240 msgr2=0x7faed00cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 --2- 172.21.15.3:0/1323209901 >> v2:172.21.15.32:3300/0 conn(0x7faed0159240 0x7faed00cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 >> v1:172.21.15.32:6789/0 conn(0x7faed00cdc50 legacy=0x7faed015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed015ce30 con 0x7faed00ce150 2023-03-03T06:25:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7faec80019f0 con 0x7faed00ce150 2023-03-03T06:25:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 >> v1:172.21.15.3:6789/0 conn(0x7faed00ce150 legacy=0x7faed00ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 --2- 172.21.15.3:0/1323209901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faebc005c30 0x7faebc006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 --2- 172.21.15.3:0/1323209901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faebc006a80 0x7faebc006ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 --2- 172.21.15.3:0/1323209901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faebc007430 0x7faebc00dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed015dd40 con 0x7faebc006a80 2023-03-03T06:25:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc00e340 con 0x7faebc007430 2023-03-03T06:25:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc00e500 con 0x7faebc005c30 2023-03-03T06:25:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed5c63700 1 --2- 172.21.15.3:0/1323209901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faebc006a80 0x7faebc006ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed6464700 1 --2- 172.21.15.3:0/1323209901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faebc005c30 0x7faebc006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.420+0000 7faed6c65700 1 --2- 172.21.15.3:0/1323209901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faebc007430 0x7faebc00dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed5c63700 1 -- 172.21.15.3:0/1323209901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faebc005c30 msgr2=0x7faebc006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed5c63700 1 --2- 172.21.15.3:0/1323209901 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faebc005c30 0x7faebc006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed5c63700 1 -- 172.21.15.3:0/1323209901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faebc007430 msgr2=0x7faebc00dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed5c63700 1 --2- 172.21.15.3:0/1323209901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faebc007430 0x7faebc00dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed5c63700 1 -- 172.21.15.3:0/1323209901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc0057e0 con 0x7faebc006a80 2023-03-03T06:25:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed6c65700 1 -- 172.21.15.3:0/1323209901 reap_dead start 2023-03-03T06:25:06.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed5c63700 1 --2- 172.21.15.3:0/1323209901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faebc006a80 0x7faebc006ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faec00017e0 tx=0x7faebc007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc21538fef67a5ca server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:06.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec00052d0 con 0x7faebc006a80 2023-03-03T06:25:06.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec0005450 con 0x7faebc006a80 2023-03-03T06:25:06.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed5462700 1 -- 172.21.15.3:0/1323209901 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec00038e0 con 0x7faebc006a80 2023-03-03T06:25:06.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed7466700 1 -- 172.21.15.3:0/1323209901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faebc006a80 msgr2=0x7faebc006ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:06.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed7466700 1 --2- 172.21.15.3:0/1323209901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faebc006a80 0x7faebc006ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faec00017e0 tx=0x7faebc007ab0 comp rx=0 tx=0).stop 2023-03-03T06:25:06.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed7466700 1 -- 172.21.15.3:0/1323209901 shutdown_connections 2023-03-03T06:25:06.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed7466700 1 --2- 172.21.15.3:0/1323209901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faebc006a80 0x7faebc006ef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:06.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.424+0000 7faed7466700 1 -- 172.21.15.3:0/1323209901 >> 172.21.15.3:0/1323209901 conn(0x7faed0069d70 msgr2=0x7faed006a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:06.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.428+0000 7faed7466700 1 -- 172.21.15.3:0/1323209901 shutdown_connections 2023-03-03T06:25:06.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.428+0000 7faed7466700 1 -- 172.21.15.3:0/1323209901 wait complete. 2023-03-03T06:25:06.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed7466700 1 Processor -- start 2023-03-03T06:25:06.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed7466700 1 -- start start 2023-03-03T06:25:06.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed7466700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faebc006a80 0x7faed00c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed7466700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed015a4d0 0x7faed00b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed7466700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed00b3d70 0x7faed013f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed7466700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00b41e0 con 0x7faed00b3d70 2023-03-03T06:25:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed7466700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed01597a0 con 0x7faed015a4d0 2023-03-03T06:25:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed7466700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed015a040 con 0x7faebc006a80 2023-03-03T06:25:06.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed6c65700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed00b3d70 0x7faed013f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:06.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed5c63700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed015a4d0 0x7faed00b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:06.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed6c65700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed00b3d70 0x7faed013f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53710/0 (socket says 172.21.15.3:53710) 2023-03-03T06:25:06.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed6c65700 1 -- 172.21.15.3:0/3020791077 learned_addr learned my addr 172.21.15.3:0/3020791077 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:06.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed5c63700 1 -- 172.21.15.3:0/3020791077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faebc006a80 msgr2=0x7faed00c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:06.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed5c63700 1 --2- 172.21.15.3:0/3020791077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faebc006a80 0x7faed00c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:06.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed5c63700 1 -- 172.21.15.3:0/3020791077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed00b3d70 msgr2=0x7faed013f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:06.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.432+0000 7faed5c63700 1 --2- 172.21.15.3:0/3020791077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed00b3d70 0x7faed013f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:06.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.436+0000 7faed5c63700 1 -- 172.21.15.3:0/3020791077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed0159610 con 0x7faed015a4d0 2023-03-03T06:25:06.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.436+0000 7faed6c65700 1 --2- 172.21.15.3:0/3020791077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed00b3d70 0x7faed013f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:06.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.436+0000 7faed5c63700 1 --2- 172.21.15.3:0/3020791077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed015a4d0 0x7faed00b3830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faec0007fd0 tx=0x7faec0001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:06.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.436+0000 7faec6ffd700 1 -- 172.21.15.3:0/3020791077 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec0001290 con 0x7faed015a4d0 2023-03-03T06:25:06.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.436+0000 7faec6ffd700 1 -- 172.21.15.3:0/3020791077 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec0007040 con 0x7faed015a4d0 2023-03-03T06:25:06.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.436+0000 7faec6ffd700 1 -- 172.21.15.3:0/3020791077 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec00073f0 con 0x7faed015a4d0 2023-03-03T06:25:06.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.436+0000 7faed7466700 1 -- 172.21.15.3:0/3020791077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed015d110 con 0x7faed015a4d0 2023-03-03T06:25:06.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.436+0000 7faed7466700 1 -- 172.21.15.3:0/3020791077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed00705e0 con 0x7faed015a4d0 2023-03-03T06:25:06.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.436+0000 7faed7466700 1 -- 172.21.15.3:0/3020791077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed0073e90 con 0x7faed015a4d0 2023-03-03T06:25:06.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.440+0000 7faec6ffd700 1 -- 172.21.15.3:0/3020791077 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7faec002d070 con 0x7faed015a4d0 2023-03-03T06:25:06.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.440+0000 7faec6ffd700 1 --2- 172.21.15.3:0/3020791077 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7faeb4064fe0 0x7faeb4067490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:06.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.440+0000 7faec6ffd700 1 -- 172.21.15.3:0/3020791077 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5199+0+0 (secure 0 0 0) 0x7faec0082220 con 0x7faed015a4d0 2023-03-03T06:25:06.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.444+0000 7faed6464700 1 --2- 172.21.15.3:0/3020791077 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7faeb4064fe0 0x7faeb4067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.444+0000 7faed6464700 1 --2- 172.21.15.3:0/3020791077 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7faeb4064fe0 0x7faeb4067490 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faed004f390 tx=0x7faecc009690 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:06.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.448+0000 7faec6ffd700 1 -- 172.21.15.3:0/3020791077 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7faec00506b0 con 0x7faed015a4d0 2023-03-03T06:25:06.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:06.632+0000 7faed7466700 1 -- 172.21.15.3:0/3020791077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faed015a180 con 0x7faed015a4d0 2023-03-03T06:25:07.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.192+0000 7faec6ffd700 1 -- 172.21.15.3:0/3020791077 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v19) v1 ==== 173+0+0 (secure 0 0 0) 0x7faed015a180 con 0x7faed015a4d0 2023-03-03T06:25:07.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.200+0000 7faed7466700 1 -- 172.21.15.3:0/3020791077 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7faeb4064fe0 msgr2=0x7faeb4067490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:07.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.200+0000 7faed7466700 1 --2- 172.21.15.3:0/3020791077 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7faeb4064fe0 0x7faeb4067490 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faed004f390 tx=0x7faecc009690 comp rx=0 tx=0).stop 2023-03-03T06:25:07.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.200+0000 7faed7466700 1 -- 172.21.15.3:0/3020791077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed015a4d0 msgr2=0x7faed00b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:07.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.200+0000 7faed7466700 1 --2- 172.21.15.3:0/3020791077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed015a4d0 0x7faed00b3830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faec0007fd0 tx=0x7faec0001120 comp rx=0 tx=0).stop 2023-03-03T06:25:07.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.200+0000 7faed7466700 1 -- 172.21.15.3:0/3020791077 shutdown_connections 2023-03-03T06:25:07.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.200+0000 7faed7466700 1 --2- 172.21.15.3:0/3020791077 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7faeb4064fe0 0x7faeb4067490 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.200+0000 7faed7466700 1 --2- 172.21.15.3:0/3020791077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faed00b3d70 0x7faed013f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.200+0000 7faed7466700 1 --2- 172.21.15.3:0/3020791077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faed015a4d0 0x7faed00b3830 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.200+0000 7faed7466700 1 --2- 172.21.15.3:0/3020791077 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faebc006a80 0x7faed00c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.200+0000 7faed7466700 1 -- 172.21.15.3:0/3020791077 >> 172.21.15.3:0/3020791077 conn(0x7faed0069d70 msgr2=0x7faed015d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:07.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.200+0000 7faed7466700 1 -- 172.21.15.3:0/3020791077 shutdown_connections 2023-03-03T06:25:07.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.200+0000 7faed7466700 1 -- 172.21.15.3:0/3020791077 wait complete. 2023-03-03T06:25:07.206 INFO:teuthology.orchestra.run.smithi003.stderr:enabled application 'rbd' on pool 'rbd' 2023-03-03T06:25:07.236 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-03-03T06:25:07.237 INFO:tasks.ceph.mds.a:Restarting daemon 2023-03-03T06:25:07.237 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-03-03T06:25:07.243 INFO:tasks.ceph.mds.a:Started 2023-03-03T06:25:07.244 INFO:tasks.ceph.mds.c:Restarting daemon 2023-03-03T06:25:07.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-03-03T06:25:07.250 INFO:tasks.ceph.mds.c:Started 2023-03-03T06:25:07.251 INFO:tasks.ceph.mds.b:Restarting daemon 2023-03-03T06:25:07.251 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-03-03T06:25:07.259 INFO:tasks.ceph.mds.b:Started 2023-03-03T06:25:07.259 INFO:tasks.ceph.mds.d:Restarting daemon 2023-03-03T06:25:07.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-03-03T06:25:07.264 INFO:tasks.ceph.mds.d:Started 2023-03-03T06:25:07.264 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-03-03T06:25:07.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:25:07.389 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-03-03T06:25:07.390 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-03-03T06:25:07.405 INFO:tasks.ceph.mds.b.smithi032.stdout:starting mds.b at 2023-03-03T06:25:07.407 INFO:tasks.ceph.mds.d.smithi032.stdout:starting mds.d at 2023-03-03T06:25:07.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.476+0000 7f1d45701700 1 Processor -- start 2023-03-03T06:25:07.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.476+0000 7f1d45701700 1 -- start start 2023-03-03T06:25:07.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.476+0000 7f1d45701700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d400b2370 0x7f1d40160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:07.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.476+0000 7f1d45701700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d400caf40 0x7f1d400ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:07.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.476+0000 7f1d45701700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d400ce1c0 con 0x7f1d40145e20 2023-03-03T06:25:07.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.476+0000 7f1d45701700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d40160f30 con 0x7f1d400caf40 2023-03-03T06:25:07.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.476+0000 7f1d45701700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d40160970 con 0x7f1d400b2370 2023-03-03T06:25:07.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d3ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d400caf40 0x7f1d400ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:07.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d3e7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1d40145e20 0x7f1d400ca830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51254/0 (socket says 172.21.15.3:51254) 2023-03-03T06:25:07.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d3e7fc700 1 -- 172.21.15.3:0/101399428 learned_addr learned my addr 172.21.15.3:0/101399428 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:07.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d3effd700 1 --2- 172.21.15.3:0/101399428 >> v2:172.21.15.32:3300/0 conn(0x7f1d400b2370 0x7f1d40160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:07.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d3ffff700 1 -- 172.21.15.3:0/101399428 >> v2:172.21.15.32:3300/0 conn(0x7f1d400b2370 msgr2=0x7f1d40160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:07.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d3ffff700 1 --2- 172.21.15.3:0/101399428 >> v2:172.21.15.32:3300/0 conn(0x7f1d400b2370 0x7f1d40160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d3ffff700 1 -- 172.21.15.3:0/101399428 >> v1:172.21.15.3:6789/0 conn(0x7f1d40145e20 legacy=0x7f1d400ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:07.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d3ffff700 1 -- 172.21.15.3:0/101399428 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d40160c50 con 0x7f1d400caf40 2023-03-03T06:25:07.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d3dffb700 1 -- 172.21.15.3:0/101399428 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272113005 0 0) 0x7f1d400ce1c0 con 0x7f1d40145e20 2023-03-03T06:25:07.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d3ffff700 1 --2- 172.21.15.3:0/101399428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d400caf40 0x7f1d400ce650 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d400ccbc0 tx=0x7f1d3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e029bcc627c053b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:07.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d3dffb700 1 -- 172.21.15.3:0/101399428 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800e070 con 0x7f1d400caf40 2023-03-03T06:25:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d3dffb700 1 -- 172.21.15.3:0/101399428 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3800bb20 con 0x7f1d400caf40 2023-03-03T06:25:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d3dffb700 1 -- 172.21.15.3:0/101399428 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38012430 con 0x7f1d400caf40 2023-03-03T06:25:07.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d45701700 1 -- 172.21.15.3:0/101399428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d400caf40 msgr2=0x7f1d400ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:07.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d45701700 1 --2- 172.21.15.3:0/101399428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d400caf40 0x7f1d400ce650 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d400ccbc0 tx=0x7f1d3800b330 comp rx=0 tx=0).stop 2023-03-03T06:25:07.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d45701700 1 -- 172.21.15.3:0/101399428 shutdown_connections 2023-03-03T06:25:07.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d45701700 1 --2- 172.21.15.3:0/101399428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d400caf40 0x7f1d400ce650 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d45701700 1 --2- 172.21.15.3:0/101399428 >> v2:172.21.15.32:3300/0 conn(0x7f1d400b2370 0x7f1d40160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d45701700 1 -- 172.21.15.3:0/101399428 >> 172.21.15.3:0/101399428 conn(0x7f1d4006b580 msgr2=0x7f1d4006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:07.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d45701700 1 -- 172.21.15.3:0/101399428 shutdown_connections 2023-03-03T06:25:07.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.480+0000 7f1d45701700 1 -- 172.21.15.3:0/101399428 wait complete. 2023-03-03T06:25:07.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.484+0000 7f1d45701700 1 Processor -- start 2023-03-03T06:25:07.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d45701700 1 -- start start 2023-03-03T06:25:07.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d45701700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d400b2370 0x7f1d400ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:07.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d45701700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d400caf40 0x7f1d400c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:07.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d45701700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40145e20 0x7f1d4015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:07.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d45701700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400c8cb0 con 0x7f1d40145e20 2023-03-03T06:25:07.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d45701700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400c8f90 con 0x7f1d400b2370 2023-03-03T06:25:07.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d45701700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400c9270 con 0x7f1d400caf40 2023-03-03T06:25:07.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d3ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40145e20 0x7f1d4015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:07.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d3ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40145e20 0x7f1d4015aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53780/0 (socket says 172.21.15.3:53780) 2023-03-03T06:25:07.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d3ffff700 1 -- 172.21.15.3:0/1209137251 learned_addr learned my addr 172.21.15.3:0/1209137251 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:07.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d3effd700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d400b2370 0x7f1d400ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:07.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d3e7fc700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d400caf40 0x7f1d400c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:07.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d3ffff700 1 -- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d400caf40 msgr2=0x7f1d400c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:07.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d3ffff700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d400caf40 0x7f1d400c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d3ffff700 1 -- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d400b2370 msgr2=0x7f1d400ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:07.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d3ffff700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d400b2370 0x7f1d400ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d3ffff700 1 -- 172.21.15.3:0/1209137251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d38009a70 con 0x7f1d40145e20 2023-03-03T06:25:07.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.488+0000 7f1d3ffff700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40145e20 0x7f1d4015aa10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d38013000 tx=0x7f1d38004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:07.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.492+0000 7f1d277fe700 1 -- 172.21.15.3:0/1209137251 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38012430 con 0x7f1d40145e20 2023-03-03T06:25:07.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.492+0000 7f1d277fe700 1 -- 172.21.15.3:0/1209137251 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d38020070 con 0x7f1d40145e20 2023-03-03T06:25:07.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.492+0000 7f1d45701700 1 -- 172.21.15.3:0/1209137251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d400b2740 con 0x7f1d40145e20 2023-03-03T06:25:07.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.492+0000 7f1d277fe700 1 -- 172.21.15.3:0/1209137251 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38004280 con 0x7f1d40145e20 2023-03-03T06:25:07.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.492+0000 7f1d45701700 1 -- 172.21.15.3:0/1209137251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d400cde50 con 0x7f1d40145e20 2023-03-03T06:25:07.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.492+0000 7f1d45701700 1 -- 172.21.15.3:0/1209137251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d40160540 con 0x7f1d40145e20 2023-03-03T06:25:07.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.496+0000 7f1d277fe700 1 -- 172.21.15.3:0/1209137251 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1d380043e0 con 0x7f1d40145e20 2023-03-03T06:25:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.496+0000 7f1d277fe700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d2c064ee0 0x7f1d2c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:07.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.496+0000 7f1d277fe700 1 -- 172.21.15.3:0/1209137251 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1d38022070 con 0x7f1d40145e20 2023-03-03T06:25:07.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.496+0000 7f1d3effd700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d2c064ee0 0x7f1d2c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:07.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.496+0000 7f1d3effd700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d2c064ee0 0x7f1d2c067390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009b50 tx=0x7f1d30009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:07.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.500+0000 7f1d277fe700 1 -- 172.21.15.3:0/1209137251 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1d38052960 con 0x7f1d40145e20 2023-03-03T06:25:07.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.680+0000 7f1d45701700 1 -- 172.21.15.3:0/1209137251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d4004e180 con 0x7f1d40145e20 2023-03-03T06:25:07.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.680+0000 7f1d277fe700 1 -- 172.21.15.3:0/1209137251 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14085 (secure 0 0 0) 0x7f1d38055f80 con 0x7f1d40145e20 2023-03-03T06:25:07.686 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:07.687 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":19,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:07.181672+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:25:07.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.684+0000 7f1d45701700 1 -- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d2c064ee0 msgr2=0x7f1d2c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:07.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.684+0000 7f1d45701700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d2c064ee0 0x7f1d2c067390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009b50 tx=0x7f1d30009240 comp rx=0 tx=0).stop 2023-03-03T06:25:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.684+0000 7f1d45701700 1 -- 172.21.15.3:0/1209137251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40145e20 msgr2=0x7f1d4015aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.684+0000 7f1d45701700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40145e20 0x7f1d4015aa10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d38013000 tx=0x7f1d38004af0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.684+0000 7f1d45701700 1 -- 172.21.15.3:0/1209137251 shutdown_connections 2023-03-03T06:25:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.684+0000 7f1d45701700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d2c064ee0 0x7f1d2c067390 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.684+0000 7f1d45701700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d40145e20 0x7f1d4015aa10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.684+0000 7f1d45701700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d400caf40 0x7f1d400c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.684+0000 7f1d45701700 1 --2- 172.21.15.3:0/1209137251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d400b2370 0x7f1d400ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.684+0000 7f1d45701700 1 -- 172.21.15.3:0/1209137251 >> 172.21.15.3:0/1209137251 conn(0x7f1d4006b580 msgr2=0x7f1d4015be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:07.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.684+0000 7f1d45701700 1 -- 172.21.15.3:0/1209137251 shutdown_connections 2023-03-03T06:25:07.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.684+0000 7f1d45701700 1 -- 172.21.15.3:0/1209137251 wait complete. 2023-03-03T06:25:07.719 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-03-03T06:25:07.720 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:25:07.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.928+0000 7f02292d7700 1 Processor -- start 2023-03-03T06:25:07.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.928+0000 7f02292d7700 1 -- start start 2023-03-03T06:25:07.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.932+0000 7f02292d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0224156ba0 0x7f0224071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:07.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.932+0000 7f02292d7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f022415d6e0 0x7f02240cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:07.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.932+0000 7f02292d7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f02240cfd80 0x7f02240d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:07.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.932+0000 7f02292d7700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02240cf3b0 con 0x7f02240cfd80 2023-03-03T06:25:07.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.932+0000 7f02292d7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0224156f70 con 0x7f022415d6e0 2023-03-03T06:25:07.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.932+0000 7f02292d7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02240cd320 con 0x7f0224156ba0 2023-03-03T06:25:07.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f02237fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f022415d6e0 0x7f02240cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:07.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f0228ad6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f02240cfd80 0x7f02240d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:07.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f0223fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0224156ba0 0x7f0224071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:07.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f02237fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f022415d6e0 0x7f02240cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:46198/0 (socket says 172.21.15.3:46198) 2023-03-03T06:25:07.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f0228ad6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f02240cfd80 0x7f02240d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53784/0 (socket says 172.21.15.3:53784) 2023-03-03T06:25:07.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f02237fe700 1 -- 172.21.15.3:0/2363047879 learned_addr learned my addr 172.21.15.3:0/2363047879 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:07.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f0223fff700 1 -- 172.21.15.3:0/2363047879 >> v2:172.21.15.32:3300/0 conn(0x7f022415d6e0 msgr2=0x7f02240cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:07.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f0223fff700 1 --2- 172.21.15.3:0/2363047879 >> v2:172.21.15.32:3300/0 conn(0x7f022415d6e0 0x7f02240cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f0223fff700 1 -- 172.21.15.3:0/2363047879 >> v2:172.21.15.3:3300/0 conn(0x7f02240cfd80 msgr2=0x7f02240d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:07.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f0223fff700 1 --2- 172.21.15.3:0/2363047879 >> v2:172.21.15.3:3300/0 conn(0x7f02240cfd80 0x7f02240d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f0223fff700 1 -- 172.21.15.3:0/2363047879 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02240cd600 con 0x7f0224156ba0 2023-03-03T06:25:07.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f0223fff700 1 --2- 172.21.15.3:0/2363047879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0224156ba0 0x7f0224071280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f02240d07b0 tx=0x7f0214009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=529981f86de316a8 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f0222ffd700 1 -- 172.21.15.3:0/2363047879 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214011830 con 0x7f0224156ba0 2023-03-03T06:25:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f0222ffd700 1 -- 172.21.15.3:0/2363047879 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0214011990 con 0x7f0224156ba0 2023-03-03T06:25:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.936+0000 7f0222ffd700 1 -- 172.21.15.3:0/2363047879 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214011d20 con 0x7f0224156ba0 2023-03-03T06:25:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.940+0000 7f02292d7700 1 -- 172.21.15.3:0/2363047879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0224156ba0 msgr2=0x7f0224071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.940+0000 7f02292d7700 1 --2- 172.21.15.3:0/2363047879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0224156ba0 0x7f0224071280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f02240d07b0 tx=0x7f0214009b30 comp rx=0 tx=0).stop 2023-03-03T06:25:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.940+0000 7f02292d7700 1 -- 172.21.15.3:0/2363047879 shutdown_connections 2023-03-03T06:25:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.940+0000 7f02292d7700 1 --2- 172.21.15.3:0/2363047879 >> v2:172.21.15.3:3300/0 conn(0x7f02240cfd80 0x7f02240d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.940+0000 7f02292d7700 1 --2- 172.21.15.3:0/2363047879 >> v2:172.21.15.32:3300/0 conn(0x7f022415d6e0 0x7f02240cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.940+0000 7f02292d7700 1 --2- 172.21.15.3:0/2363047879 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0224156ba0 0x7f0224071280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.940+0000 7f02292d7700 1 -- 172.21.15.3:0/2363047879 >> 172.21.15.3:0/2363047879 conn(0x7f022406e5c0 msgr2=0x7f022406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.940+0000 7f02292d7700 1 -- 172.21.15.3:0/2363047879 shutdown_connections 2023-03-03T06:25:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.940+0000 7f02292d7700 1 -- 172.21.15.3:0/2363047879 wait complete. 2023-03-03T06:25:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.940+0000 7f02292d7700 1 Processor -- start 2023-03-03T06:25:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02292d7700 1 -- start start 2023-03-03T06:25:07.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02292d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02240cfd80 0x7f02240b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:07.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02292d7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224156ba0 0x7f02240b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02292d7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f022415d6e0 0x7f0224155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02292d7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02240b7900 con 0x7f0224156ba0 2023-03-03T06:25:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02292d7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0224155fc0 con 0x7f02240cfd80 2023-03-03T06:25:07.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02292d7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02241f5cf0 con 0x7f022415d6e0 2023-03-03T06:25:07.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02237fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224156ba0 0x7f02240b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:07.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02237fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224156ba0 0x7f02240b7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53792/0 (socket says 172.21.15.3:53792) 2023-03-03T06:25:07.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02237fe700 1 -- 172.21.15.3:0/1370406520 learned_addr learned my addr 172.21.15.3:0/1370406520 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:07.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f0223fff700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02240cfd80 0x7f02240b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:07.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f0228ad6700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f022415d6e0 0x7f0224155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:07.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02237fe700 1 -- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f022415d6e0 msgr2=0x7f0224155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:07.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02237fe700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f022415d6e0 0x7f0224155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02237fe700 1 -- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02240cfd80 msgr2=0x7f02240b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:07.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02237fe700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02240cfd80 0x7f02240b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:07.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.944+0000 7f02237fe700 1 -- 172.21.15.3:0/1370406520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02140094b0 con 0x7f0224156ba0 2023-03-03T06:25:07.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.948+0000 7f02237fe700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224156ba0 0x7f02240b7c80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f021800e8f0 tx=0x7f021800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:07.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.948+0000 7f0220ff9700 1 -- 172.21.15.3:0/1370406520 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218013680 con 0x7f0224156ba0 2023-03-03T06:25:07.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.948+0000 7f0220ff9700 1 -- 172.21.15.3:0/1370406520 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0218013800 con 0x7f0224156ba0 2023-03-03T06:25:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.948+0000 7f0220ff9700 1 -- 172.21.15.3:0/1370406520 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218013bb0 con 0x7f0224156ba0 2023-03-03T06:25:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.948+0000 7f02292d7700 1 -- 172.21.15.3:0/1370406520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02240717c0 con 0x7f0224156ba0 2023-03-03T06:25:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.948+0000 7f02292d7700 1 -- 172.21.15.3:0/1370406520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02240cd8e0 con 0x7f0224156ba0 2023-03-03T06:25:07.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.948+0000 7f02292d7700 1 -- 172.21.15.3:0/1370406520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02240cef40 con 0x7f0224156ba0 2023-03-03T06:25:07.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.952+0000 7f0220ff9700 1 -- 172.21.15.3:0/1370406520 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f0218019030 con 0x7f0224156ba0 2023-03-03T06:25:07.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.952+0000 7f0220ff9700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0210064ee0 0x7f0210067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:07.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.952+0000 7f0220ff9700 1 -- 172.21.15.3:0/1370406520 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0218081a20 con 0x7f0224156ba0 2023-03-03T06:25:07.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.956+0000 7f0223fff700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0210064ee0 0x7f0210067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:07.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.956+0000 7f0223fff700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0210064ee0 0x7f0210067390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0214000c00 tx=0x7f0214008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:07.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:07.960+0000 7f0220ff9700 1 -- 172.21.15.3:0/1370406520 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f021804fe20 con 0x7f0224156ba0 2023-03-03T06:25:08.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.140+0000 7f02292d7700 1 -- 172.21.15.3:0/1370406520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0224073e90 con 0x7f0224156ba0 2023-03-03T06:25:08.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.140+0000 7f0220ff9700 1 -- 172.21.15.3:0/1370406520 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0218053440 con 0x7f0224156ba0 2023-03-03T06:25:08.144 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:25:08.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.144+0000 7f02292d7700 1 -- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0210064ee0 msgr2=0x7f0210067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:08.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.144+0000 7f02292d7700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0210064ee0 0x7f0210067390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0214000c00 tx=0x7f0214008030 comp rx=0 tx=0).stop 2023-03-03T06:25:08.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.144+0000 7f02292d7700 1 -- 172.21.15.3:0/1370406520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224156ba0 msgr2=0x7f02240b7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:08.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.144+0000 7f02292d7700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224156ba0 0x7f02240b7c80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f021800e8f0 tx=0x7f021800ec00 comp rx=0 tx=0).stop 2023-03-03T06:25:08.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.144+0000 7f02292d7700 1 -- 172.21.15.3:0/1370406520 shutdown_connections 2023-03-03T06:25:08.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.144+0000 7f02292d7700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0210064ee0 0x7f0210067390 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.144+0000 7f02292d7700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f022415d6e0 0x7f0224155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.144+0000 7f02292d7700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0224156ba0 0x7f02240b7c80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.144+0000 7f02292d7700 1 --2- 172.21.15.3:0/1370406520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02240cfd80 0x7f02240b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.144+0000 7f02292d7700 1 -- 172.21.15.3:0/1370406520 >> 172.21.15.3:0/1370406520 conn(0x7f022406e5c0 msgr2=0x7f022415f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:08.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.144+0000 7f02292d7700 1 -- 172.21.15.3:0/1370406520 shutdown_connections 2023-03-03T06:25:08.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.144+0000 7f02292d7700 1 -- 172.21.15.3:0/1370406520 wait complete. 2023-03-03T06:25:08.182 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:25:08.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.400+0000 7f12a8dc3700 1 Processor -- start 2023-03-03T06:25:08.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.400+0000 7f12a8dc3700 1 -- start start 2023-03-03T06:25:08.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a8dc3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12a415c370 0x7f12a406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a8dc3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f12a40cb520 0x7f12a415d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a8dc3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12a40b07c0 0x7f12a40b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a8dc3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a40ce520 con 0x7f12a415c370 2023-03-03T06:25:08.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a8dc3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a40b1160 con 0x7f12a40cb520 2023-03-03T06:25:08.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a8dc3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a415d940 con 0x7f12a40b07c0 2023-03-03T06:25:08.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12a415c370 0x7f12a406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f12a415c370 0x7f12a406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53798/0 (socket says 172.21.15.3:53798) 2023-03-03T06:25:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a37fe700 1 -- 172.21.15.3:0/2350976838 learned_addr learned my addr 172.21.15.3:0/2350976838 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a2ffd700 1 --2- 172.21.15.3:0/2350976838 >> v2:172.21.15.32:3300/0 conn(0x7f12a40cb520 0x7f12a415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a3fff700 1 --2- 172.21.15.3:0/2350976838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12a40b07c0 0x7f12a40b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a37fe700 1 -- 172.21.15.3:0/2350976838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12a40b07c0 msgr2=0x7f12a40b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a37fe700 1 --2- 172.21.15.3:0/2350976838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12a40b07c0 0x7f12a40b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a37fe700 1 -- 172.21.15.3:0/2350976838 >> v2:172.21.15.32:3300/0 conn(0x7f12a40cb520 msgr2=0x7f12a415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a37fe700 1 --2- 172.21.15.3:0/2350976838 >> v2:172.21.15.32:3300/0 conn(0x7f12a40cb520 0x7f12a415d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a37fe700 1 -- 172.21.15.3:0/2350976838 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a40ccbc0 con 0x7f12a415c370 2023-03-03T06:25:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.404+0000 7f12a2ffd700 1 --2- 172.21.15.3:0/2350976838 >> v2:172.21.15.32:3300/0 conn(0x7f12a40cb520 0x7f12a415d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:08.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.408+0000 7f12a37fe700 1 --2- 172.21.15.3:0/2350976838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a415c370 0x7f12a406c090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f12a40b13e0 tx=0x7f1294009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f698c79778a068 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:08.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.408+0000 7f12a27fc700 1 -- 172.21.15.3:0/2350976838 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129400b8a0 con 0x7f12a415c370 2023-03-03T06:25:08.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.408+0000 7f12a27fc700 1 -- 172.21.15.3:0/2350976838 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129400ba00 con 0x7f12a415c370 2023-03-03T06:25:08.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.408+0000 7f12a27fc700 1 -- 172.21.15.3:0/2350976838 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129400bd90 con 0x7f12a415c370 2023-03-03T06:25:08.411 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.408+0000 7f12a8dc3700 1 -- 172.21.15.3:0/2350976838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a415c370 msgr2=0x7f12a406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:08.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.408+0000 7f12a8dc3700 1 --2- 172.21.15.3:0/2350976838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a415c370 0x7f12a406c090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f12a40b13e0 tx=0x7f1294009bf0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.408+0000 7f12a8dc3700 1 -- 172.21.15.3:0/2350976838 shutdown_connections 2023-03-03T06:25:08.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.408+0000 7f12a8dc3700 1 --2- 172.21.15.3:0/2350976838 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12a40b07c0 0x7f12a40b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.408+0000 7f12a8dc3700 1 --2- 172.21.15.3:0/2350976838 >> v2:172.21.15.32:3300/0 conn(0x7f12a40cb520 0x7f12a415d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.408+0000 7f12a8dc3700 1 --2- 172.21.15.3:0/2350976838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a415c370 0x7f12a406c090 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.408+0000 7f12a8dc3700 1 -- 172.21.15.3:0/2350976838 >> 172.21.15.3:0/2350976838 conn(0x7f12a40607c0 msgr2=0x7f12a4060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:08.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.408+0000 7f12a8dc3700 1 -- 172.21.15.3:0/2350976838 shutdown_connections 2023-03-03T06:25:08.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.408+0000 7f12a8dc3700 1 -- 172.21.15.3:0/2350976838 wait complete. 2023-03-03T06:25:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.416+0000 7f12a8dc3700 1 Processor -- start 2023-03-03T06:25:08.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.416+0000 7f12a8dc3700 1 -- start start 2023-03-03T06:25:08.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.416+0000 7f12a8dc3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12a40b07c0 0x7f12a4159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.416+0000 7f12a8dc3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12a40cb520 0x7f12a40c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.416+0000 7f12a8dc3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a415c370 0x7f12a40c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.416+0000 7f12a8dc3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a415a2d0 con 0x7f12a415c370 2023-03-03T06:25:08.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.416+0000 7f12a8dc3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40c61b0 con 0x7f12a40cb520 2023-03-03T06:25:08.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.416+0000 7f12a8dc3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40c6490 con 0x7f12a40b07c0 2023-03-03T06:25:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.416+0000 7f12a3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a415c370 0x7f12a40c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.416+0000 7f12a3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a415c370 0x7f12a40c9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53802/0 (socket says 172.21.15.3:53802) 2023-03-03T06:25:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.416+0000 7f12a3fff700 1 -- 172.21.15.3:0/3374933602 learned_addr learned my addr 172.21.15.3:0/3374933602 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.416+0000 7f12a37fe700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12a40b07c0 0x7f12a4159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.416+0000 7f12a2ffd700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12a40cb520 0x7f12a40c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.420+0000 7f12a3fff700 1 -- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12a40b07c0 msgr2=0x7f12a4159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:08.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.420+0000 7f12a3fff700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12a40b07c0 0x7f12a4159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.420+0000 7f12a3fff700 1 -- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12a40cb520 msgr2=0x7f12a40c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:08.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.420+0000 7f12a3fff700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12a40cb520 0x7f12a40c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.420+0000 7f12a3fff700 1 -- 172.21.15.3:0/3374933602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1294009570 con 0x7f12a415c370 2023-03-03T06:25:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.420+0000 7f12a3fff700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a415c370 0x7f12a40c9c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f129c007d20 tx=0x7f129c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.420+0000 7f128bfff700 1 -- 172.21.15.3:0/3374933602 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c00ed30 con 0x7f12a415c370 2023-03-03T06:25:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.420+0000 7f128bfff700 1 -- 172.21.15.3:0/3374933602 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c007e70 con 0x7f12a415c370 2023-03-03T06:25:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.420+0000 7f128bfff700 1 -- 172.21.15.3:0/3374933602 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c014640 con 0x7f12a415c370 2023-03-03T06:25:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.420+0000 7f12a8dc3700 1 -- 172.21.15.3:0/3374933602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a40cc500 con 0x7f12a415c370 2023-03-03T06:25:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.420+0000 7f12a8dc3700 1 -- 172.21.15.3:0/3374933602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a40cbae0 con 0x7f12a415c370 2023-03-03T06:25:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.424+0000 7f12a8dc3700 1 -- 172.21.15.3:0/3374933602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a40cc7e0 con 0x7f12a415c370 2023-03-03T06:25:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.424+0000 7f128bfff700 1 -- 172.21.15.3:0/3374933602 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f129c012050 con 0x7f12a415c370 2023-03-03T06:25:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.428+0000 7f128bfff700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1290064ee0 0x7f1290067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.428+0000 7f128bfff700 1 -- 172.21.15.3:0/3374933602 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f129c082160 con 0x7f12a415c370 2023-03-03T06:25:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.428+0000 7f12a37fe700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1290064ee0 0x7f1290067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.428+0000 7f12a37fe700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1290064ee0 0x7f1290067390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1294000c00 tx=0x7f1294008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:08.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.432+0000 7f128bfff700 1 -- 172.21.15.3:0/3374933602 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f129c050480 con 0x7f12a415c370 2023-03-03T06:25:08.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.604+0000 7f12a8dc3700 1 -- 172.21.15.3:0/3374933602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f12a40b16a0 con 0x7f12a415c370 2023-03-03T06:25:08.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.608+0000 7f128bfff700 1 -- 172.21.15.3:0/3374933602 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f12a40b16a0 con 0x7f12a415c370 2023-03-03T06:25:08.610 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:25:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.608+0000 7f12a8dc3700 1 -- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1290064ee0 msgr2=0x7f1290067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.608+0000 7f12a8dc3700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1290064ee0 0x7f1290067390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1294000c00 tx=0x7f1294008030 comp rx=0 tx=0).stop 2023-03-03T06:25:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.608+0000 7f12a8dc3700 1 -- 172.21.15.3:0/3374933602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a415c370 msgr2=0x7f12a40c9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.608+0000 7f12a8dc3700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a415c370 0x7f12a40c9c70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f129c007d20 tx=0x7f129c00e3c0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.612+0000 7f12a8dc3700 1 -- 172.21.15.3:0/3374933602 shutdown_connections 2023-03-03T06:25:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.612+0000 7f12a8dc3700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1290064ee0 0x7f1290067390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.612+0000 7f12a8dc3700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12a415c370 0x7f12a40c9c70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.612+0000 7f12a8dc3700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12a40cb520 0x7f12a40c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.612+0000 7f12a8dc3700 1 --2- 172.21.15.3:0/3374933602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f12a40b07c0 0x7f12a4159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.612+0000 7f12a8dc3700 1 -- 172.21.15.3:0/3374933602 >> 172.21.15.3:0/3374933602 conn(0x7f12a40607c0 msgr2=0x7f12a40ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.612+0000 7f12a8dc3700 1 -- 172.21.15.3:0/3374933602 shutdown_connections 2023-03-03T06:25:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.612+0000 7f12a8dc3700 1 -- 172.21.15.3:0/3374933602 wait complete. 2023-03-03T06:25:08.649 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-03T06:25:08.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.852+0000 7f5e29639700 1 Processor -- start 2023-03-03T06:25:08.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.852+0000 7f5e29639700 1 -- start start 2023-03-03T06:25:08.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e29639700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e24153670 0x7f5e24071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e29639700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e241608d0 0x7f5e240cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e29639700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e240cfb60 0x7f5e240d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e29639700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e24160440 con 0x7f5e24153670 2023-03-03T06:25:08.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e29639700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e240cffe0 con 0x7f5e241608d0 2023-03-03T06:25:08.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e29639700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240cd3e0 con 0x7f5e240cfb60 2023-03-03T06:25:08.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e237fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e241608d0 0x7f5e240cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e23fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5e24153670 0x7f5e24071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e237fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e241608d0 0x7f5e240cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:46242/0 (socket says 172.21.15.3:46242) 2023-03-03T06:25:08.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e237fe700 1 -- 172.21.15.3:0/3526830230 learned_addr learned my addr 172.21.15.3:0/3526830230 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:08.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e28e38700 1 --2- 172.21.15.3:0/3526830230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e240cfb60 0x7f5e240d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e237fe700 1 -- 172.21.15.3:0/3526830230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e240cfb60 msgr2=0x7f5e240d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:08.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e237fe700 1 --2- 172.21.15.3:0/3526830230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e240cfb60 0x7f5e240d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e237fe700 1 -- 172.21.15.3:0/3526830230 >> v2:172.21.15.3:3300/0 conn(0x7f5e24153670 msgr2=0x7f5e24071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:08.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e237fe700 1 --2- 172.21.15.3:0/3526830230 >> v2:172.21.15.3:3300/0 conn(0x7f5e24153670 0x7f5e24071350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e237fe700 1 -- 172.21.15.3:0/3526830230 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240cd6c0 con 0x7f5e241608d0 2023-03-03T06:25:08.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e23fff700 1 --2- 172.21.15.3:0/3526830230 >> v2:172.21.15.3:3300/0 conn(0x7f5e24153670 0x7f5e24071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:08.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.856+0000 7f5e237fe700 1 --2- 172.21.15.3:0/3526830230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e241608d0 0x7f5e240cf620 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009910 tx=0x7f5e18009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8102c2d53481a072 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.860+0000 7f5e22ffd700 1 -- 172.21.15.3:0/3526830230 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e18011830 con 0x7f5e241608d0 2023-03-03T06:25:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.860+0000 7f5e22ffd700 1 -- 172.21.15.3:0/3526830230 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e18011990 con 0x7f5e241608d0 2023-03-03T06:25:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.860+0000 7f5e22ffd700 1 -- 172.21.15.3:0/3526830230 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e18011d40 con 0x7f5e241608d0 2023-03-03T06:25:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.860+0000 7f5e29639700 1 -- 172.21.15.3:0/3526830230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e241608d0 msgr2=0x7f5e240cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.860+0000 7f5e29639700 1 --2- 172.21.15.3:0/3526830230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e241608d0 0x7f5e240cf620 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009910 tx=0x7f5e18009c20 comp rx=0 tx=0).stop 2023-03-03T06:25:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.860+0000 7f5e29639700 1 -- 172.21.15.3:0/3526830230 shutdown_connections 2023-03-03T06:25:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.860+0000 7f5e29639700 1 --2- 172.21.15.3:0/3526830230 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e240cfb60 0x7f5e240d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.860+0000 7f5e29639700 1 --2- 172.21.15.3:0/3526830230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e241608d0 0x7f5e240cf620 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.860+0000 7f5e29639700 1 --2- 172.21.15.3:0/3526830230 >> v2:172.21.15.3:3300/0 conn(0x7f5e24153670 0x7f5e24071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.860+0000 7f5e29639700 1 -- 172.21.15.3:0/3526830230 >> 172.21.15.3:0/3526830230 conn(0x7f5e2406e5c0 msgr2=0x7f5e2406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.860+0000 7f5e29639700 1 -- 172.21.15.3:0/3526830230 shutdown_connections 2023-03-03T06:25:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.860+0000 7f5e29639700 1 -- 172.21.15.3:0/3526830230 wait complete. 2023-03-03T06:25:08.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.864+0000 7f5e29639700 1 Processor -- start 2023-03-03T06:25:08.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.864+0000 7f5e29639700 1 -- start start 2023-03-03T06:25:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.864+0000 7f5e29639700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e240cfb60 0x7f5e240b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.864+0000 7f5e29639700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24153670 0x7f5e240ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.864+0000 7f5e29639700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e241608d0 0x7f5e240caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.864+0000 7f5e29639700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240b23c0 con 0x7f5e24153670 2023-03-03T06:25:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.864+0000 7f5e29639700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240b3220 con 0x7f5e241608d0 2023-03-03T06:25:08.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.864+0000 7f5e29639700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240b2610 con 0x7f5e240cfb60 2023-03-03T06:25:08.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e237fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24153670 0x7f5e240ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e23fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e240cfb60 0x7f5e240b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e237fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24153670 0x7f5e240ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53824/0 (socket says 172.21.15.3:53824) 2023-03-03T06:25:08.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e237fe700 1 -- 172.21.15.3:0/2671280369 learned_addr learned my addr 172.21.15.3:0/2671280369 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:08.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e28e38700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e241608d0 0x7f5e240caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e23fff700 1 -- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e241608d0 msgr2=0x7f5e240caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:08.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e23fff700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e241608d0 0x7f5e240caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e23fff700 1 -- 172.21.15.3:0/2671280369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24153670 msgr2=0x7f5e240ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:08.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e23fff700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24153670 0x7f5e240ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:08.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e23fff700 1 -- 172.21.15.3:0/2671280369 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e140094b0 con 0x7f5e240cfb60 2023-03-03T06:25:08.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e237fe700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24153670 0x7f5e240ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:08.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e23fff700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e240cfb60 0x7f5e240b36b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400e9e0 tx=0x7f5e1400ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:08.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e20ff9700 1 -- 172.21.15.3:0/2671280369 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1400ee80 con 0x7f5e240cfb60 2023-03-03T06:25:08.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e20ff9700 1 -- 172.21.15.3:0/2671280369 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1400c850 con 0x7f5e240cfb60 2023-03-03T06:25:08.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e29639700 1 -- 172.21.15.3:0/2671280369 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e18009570 con 0x7f5e240cfb60 2023-03-03T06:25:08.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e20ff9700 1 -- 172.21.15.3:0/2671280369 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1400cc00 con 0x7f5e240cfb60 2023-03-03T06:25:08.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.868+0000 7f5e29639700 1 -- 172.21.15.3:0/2671280369 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e24071890 con 0x7f5e240cfb60 2023-03-03T06:25:08.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.872+0000 7f5e29639700 1 -- 172.21.15.3:0/2671280369 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e241600d0 con 0x7f5e240cfb60 2023-03-03T06:25:08.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.876+0000 7f5e20ff9700 1 -- 172.21.15.3:0/2671280369 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5e14019030 con 0x7f5e240cfb60 2023-03-03T06:25:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.876+0000 7f5e20ff9700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5e10064ee0 0x7f5e10067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.876+0000 7f5e20ff9700 1 -- 172.21.15.3:0/2671280369 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5e14082160 con 0x7f5e240cfb60 2023-03-03T06:25:08.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.876+0000 7f5e237fe700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5e10064ee0 0x7f5e10067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:08.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.876+0000 7f5e237fe700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5e10064ee0 0x7f5e10067390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e241f9550 tx=0x7f5e18008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:08.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:08.884+0000 7f5e20ff9700 1 -- 172.21.15.3:0/2671280369 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5e140506f0 con 0x7f5e240cfb60 2023-03-03T06:25:09.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.096+0000 7f5e29639700 1 -- 172.21.15.3:0/2671280369 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e240cd9a0 con 0x7f5e240cfb60 2023-03-03T06:25:09.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.096+0000 7f5e20ff9700 1 -- 172.21.15.3:0/2671280369 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3513 (secure 0 0 0) 0x7f5e14017dd0 con 0x7f5e240cfb60 2023-03-03T06:25:09.100 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:09.100 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2744699972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2744699972},{"type":"v1","addr":"172.21.15.3:6834","nonce":2744699972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/708054272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":708054272},{"type":"v1","addr":"172.21.15.3:6835","nonce":708054272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2935923413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2935923413},{"type":"v1","addr":"172.21.15.32:6837","nonce":2935923413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":54179,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/691220172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":691220172},{"type":"v1","addr":"172.21.15.32:6836","nonce":691220172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-03-03T06:25:09.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.100+0000 7f5e29639700 1 -- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5e10064ee0 msgr2=0x7f5e10067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:09.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.100+0000 7f5e29639700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5e10064ee0 0x7f5e10067390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e241f9550 tx=0x7f5e18008030 comp rx=0 tx=0).stop 2023-03-03T06:25:09.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.100+0000 7f5e29639700 1 -- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e240cfb60 msgr2=0x7f5e240b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:09.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.100+0000 7f5e29639700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e240cfb60 0x7f5e240b36b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400e9e0 tx=0x7f5e1400ecf0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.100+0000 7f5e29639700 1 -- 172.21.15.3:0/2671280369 shutdown_connections 2023-03-03T06:25:09.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.100+0000 7f5e29639700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5e10064ee0 0x7f5e10067390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.100+0000 7f5e29639700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e241608d0 0x7f5e240caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.100+0000 7f5e29639700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24153670 0x7f5e240ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.100+0000 7f5e29639700 1 --2- 172.21.15.3:0/2671280369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5e240cfb60 0x7f5e240b36b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.100+0000 7f5e29639700 1 -- 172.21.15.3:0/2671280369 >> 172.21.15.3:0/2671280369 conn(0x7f5e2406e5c0 msgr2=0x7f5e2415ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:09.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.100+0000 7f5e29639700 1 -- 172.21.15.3:0/2671280369 shutdown_connections 2023-03-03T06:25:09.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.100+0000 7f5e29639700 1 -- 172.21.15.3:0/2671280369 wait complete. 2023-03-03T06:25:09.107 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 3 2023-03-03T06:25:09.139 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:25:09.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.344+0000 7f710d284700 1 Processor -- start 2023-03-03T06:25:09.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.344+0000 7f710d284700 1 -- start start 2023-03-03T06:25:09.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f710d284700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71080adf20 0x7f71080b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:09.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f710d284700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f710815e310 0x7f71080cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:09.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f710d284700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71080ae2f0 con 0x7f710815dec0 2023-03-03T06:25:09.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f710d284700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71080c9a20 con 0x7f710815e310 2023-03-03T06:25:09.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f710d284700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71080c9fe0 con 0x7f71080adf20 2023-03-03T06:25:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f71077fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f710815dec0 0x7f71080ce710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51262/0 (socket says 172.21.15.3:51262) 2023-03-03T06:25:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f71077fe700 1 -- 172.21.15.3:0/1280230859 learned_addr learned my addr 172.21.15.3:0/1280230859 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f7107fff700 1 --2- 172.21.15.3:0/1280230859 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71080adf20 0x7f71080b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f710ca83700 1 --2- 172.21.15.3:0/1280230859 >> v2:172.21.15.32:3300/0 conn(0x7f710815e310 0x7f71080cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f7106ffd700 1 -- 172.21.15.3:0/1280230859 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914950550 0 0) 0x7f71080ae2f0 con 0x7f710815dec0 2023-03-03T06:25:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f7106ffd700 1 -- 172.21.15.3:0/1280230859 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70f00035d0 con 0x7f710815dec0 2023-03-03T06:25:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f710ca83700 1 -- 172.21.15.3:0/1280230859 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71080adf20 msgr2=0x7f71080b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f710ca83700 1 --2- 172.21.15.3:0/1280230859 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71080adf20 0x7f71080b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f710ca83700 1 -- 172.21.15.3:0/1280230859 >> v1:172.21.15.3:6789/0 conn(0x7f710815dec0 legacy=0x7f71080ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.348+0000 7f710ca83700 1 -- 172.21.15.3:0/1280230859 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710815a7d0 con 0x7f710815e310 2023-03-03T06:25:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.352+0000 7f710ca83700 1 --2- 172.21.15.3:0/1280230859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710815e310 0x7f71080cb480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7100009d90 tx=0x7f710000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffe34724451e16aa server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:09.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.352+0000 7f7106ffd700 1 -- 172.21.15.3:0/1280230859 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710000e070 con 0x7f710815e310 2023-03-03T06:25:09.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.352+0000 7f7106ffd700 1 -- 172.21.15.3:0/1280230859 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710000bb20 con 0x7f710815e310 2023-03-03T06:25:09.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.352+0000 7f7106ffd700 1 -- 172.21.15.3:0/1280230859 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100012430 con 0x7f710815e310 2023-03-03T06:25:09.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.352+0000 7f710d284700 1 -- 172.21.15.3:0/1280230859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710815e310 msgr2=0x7f71080cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.352+0000 7f710d284700 1 --2- 172.21.15.3:0/1280230859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710815e310 0x7f71080cb480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7100009d90 tx=0x7f710000b330 comp rx=0 tx=0).stop 2023-03-03T06:25:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.352+0000 7f710d284700 1 -- 172.21.15.3:0/1280230859 shutdown_connections 2023-03-03T06:25:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.352+0000 7f710d284700 1 --2- 172.21.15.3:0/1280230859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710815e310 0x7f71080cb480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.352+0000 7f710d284700 1 --2- 172.21.15.3:0/1280230859 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71080adf20 0x7f71080b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.352+0000 7f710d284700 1 -- 172.21.15.3:0/1280230859 >> 172.21.15.3:0/1280230859 conn(0x7f7108070a30 msgr2=0x7f7108070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.352+0000 7f710d284700 1 -- 172.21.15.3:0/1280230859 shutdown_connections 2023-03-03T06:25:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.352+0000 7f710d284700 1 -- 172.21.15.3:0/1280230859 wait complete. 2023-03-03T06:25:09.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.356+0000 7f710d284700 1 Processor -- start 2023-03-03T06:25:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.356+0000 7f710d284700 1 -- start start 2023-03-03T06:25:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f710d284700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71080adf20 0x7f71081f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f710d284700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710815dec0 0x7f710815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f710d284700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f710815e310 0x7f71080c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:09.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f710d284700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71080b2700 con 0x7f71080adf20 2023-03-03T06:25:09.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f710d284700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71081f28d0 con 0x7f710815dec0 2023-03-03T06:25:09.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f710d284700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710815ca00 con 0x7f710815e310 2023-03-03T06:25:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f7107fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71080adf20 0x7f71081f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f7107fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71080adf20 0x7f71081f2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53832/0 (socket says 172.21.15.3:53832) 2023-03-03T06:25:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f7107fff700 1 -- 172.21.15.3:0/831206336 learned_addr learned my addr 172.21.15.3:0/831206336 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f71077fe700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710815dec0 0x7f710815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f710ca83700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f710815e310 0x7f71080c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f7107fff700 1 -- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f710815e310 msgr2=0x7f71080c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f7107fff700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f710815e310 0x7f71080c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f7107fff700 1 -- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710815dec0 msgr2=0x7f710815b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f7107fff700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710815dec0 0x7f710815b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f7107fff700 1 -- 172.21.15.3:0/831206336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7100009a70 con 0x7f71080adf20 2023-03-03T06:25:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f7107fff700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71080adf20 0x7f71081f2390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f70f800e9d0 tx=0x7f70f800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f7104ff9700 1 -- 172.21.15.3:0/831206336 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8009720 con 0x7f71080adf20 2023-03-03T06:25:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f7104ff9700 1 -- 172.21.15.3:0/831206336 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f800c850 con 0x7f71080adf20 2023-03-03T06:25:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f710d284700 1 -- 172.21.15.3:0/831206336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710815ebf0 con 0x7f71080adf20 2023-03-03T06:25:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f7104ff9700 1 -- 172.21.15.3:0/831206336 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f800cbc0 con 0x7f71080adf20 2023-03-03T06:25:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.360+0000 7f710d284700 1 -- 172.21.15.3:0/831206336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710815e910 con 0x7f71080adf20 2023-03-03T06:25:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.364+0000 7f710d284700 1 -- 172.21.15.3:0/831206336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71080c9c70 con 0x7f71080adf20 2023-03-03T06:25:09.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.368+0000 7f7104ff9700 1 -- 172.21.15.3:0/831206336 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f70f8019030 con 0x7f71080adf20 2023-03-03T06:25:09.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.368+0000 7f7104ff9700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f70f4064ee0 0x7f70f4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:09.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.368+0000 7f7104ff9700 1 -- 172.21.15.3:0/831206336 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f70f8081c80 con 0x7f71080adf20 2023-03-03T06:25:09.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.368+0000 7f71077fe700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f70f4064ee0 0x7f70f4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:09.373 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.372+0000 7f71077fe700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f70f4064ee0 0x7f70f4067390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71080b1b50 tx=0x7f70fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:09.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.376+0000 7f7104ff9700 1 -- 172.21.15.3:0/831206336 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f70f8050080 con 0x7f71080adf20 2023-03-03T06:25:09.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.556+0000 7f710d284700 1 -- 172.21.15.3:0/831206336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7108073e90 con 0x7f71080adf20 2023-03-03T06:25:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.560+0000 7f7104ff9700 1 -- 172.21.15.3:0/831206336 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+14072 (secure 0 0 0) 0x7f70f80536a0 con 0x7f71080adf20 2023-03-03T06:25:09.564 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:08.190987+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:25:09.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.564+0000 7f710d284700 1 -- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f70f4064ee0 msgr2=0x7f70f4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.564+0000 7f710d284700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f70f4064ee0 0x7f70f4067390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f71080b1b50 tx=0x7f70fc009040 comp rx=0 tx=0).stop 2023-03-03T06:25:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.564+0000 7f710d284700 1 -- 172.21.15.3:0/831206336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71080adf20 msgr2=0x7f71081f2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.564+0000 7f710d284700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71080adf20 0x7f71081f2390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f70f800e9d0 tx=0x7f70f800ed90 comp rx=0 tx=0).stop 2023-03-03T06:25:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.564+0000 7f710d284700 1 -- 172.21.15.3:0/831206336 shutdown_connections 2023-03-03T06:25:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.564+0000 7f710d284700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f70f4064ee0 0x7f70f4067390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.564+0000 7f710d284700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f710815e310 0x7f71080c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.564+0000 7f710d284700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f710815dec0 0x7f710815b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.564+0000 7f710d284700 1 --2- 172.21.15.3:0/831206336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71080adf20 0x7f71081f2390 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.564+0000 7f710d284700 1 -- 172.21.15.3:0/831206336 >> 172.21.15.3:0/831206336 conn(0x7f7108070a30 msgr2=0x7f71080cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.568+0000 7f710d284700 1 -- 172.21.15.3:0/831206336 shutdown_connections 2023-03-03T06:25:09.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.568+0000 7f710d284700 1 -- 172.21.15.3:0/831206336 wait complete. 2023-03-03T06:25:09.604 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-03-03T06:25:09.605 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:25:09.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.812+0000 7f3640f23700 1 Processor -- start 2023-03-03T06:25:09.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.812+0000 7f3640f23700 1 -- start start 2023-03-03T06:25:09.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f3640f23700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f363c0b4d90 0x7f363c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:09.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f3640f23700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f363c0cd7d0 0x7f363c0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:09.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f3640f23700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363c0cd2f0 con 0x7f363c15faf0 2023-03-03T06:25:09.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f3640f23700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c1609d0 con 0x7f363c0b4d90 2023-03-03T06:25:09.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f3640f23700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c0cf520 con 0x7f363c0cd7d0 2023-03-03T06:25:09.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363b7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f363c15faf0 0x7f363c0b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51270/0 (socket says 172.21.15.3:51270) 2023-03-03T06:25:09.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363b7fe700 1 -- 172.21.15.3:0/134500485 learned_addr learned my addr 172.21.15.3:0/134500485 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:09.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363affd700 1 --2- 172.21.15.3:0/134500485 >> v2:172.21.15.3:3300/0 conn(0x7f363c0b4d90 0x7f363c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:09.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363bfff700 1 --2- 172.21.15.3:0/134500485 >> v2:172.21.15.32:3300/0 conn(0x7f363c0cd7d0 0x7f363c0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:09.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363a7fc700 1 -- 172.21.15.3:0/134500485 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085701840 0 0) 0x7f363c0cd2f0 con 0x7f363c15faf0 2023-03-03T06:25:09.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363a7fc700 1 -- 172.21.15.3:0/134500485 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36240035d0 con 0x7f363c15faf0 2023-03-03T06:25:09.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363bfff700 1 -- 172.21.15.3:0/134500485 >> v2:172.21.15.3:3300/0 conn(0x7f363c0b4d90 msgr2=0x7f363c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:09.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363bfff700 1 --2- 172.21.15.3:0/134500485 >> v2:172.21.15.3:3300/0 conn(0x7f363c0b4d90 0x7f363c0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363bfff700 1 -- 172.21.15.3:0/134500485 >> v1:172.21.15.3:6789/0 conn(0x7f363c15faf0 legacy=0x7f363c0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:09.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363bfff700 1 -- 172.21.15.3:0/134500485 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0cf240 con 0x7f363c0cd7d0 2023-03-03T06:25:09.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363bfff700 1 --2- 172.21.15.3:0/134500485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f363c0cd7d0 0x7f363c0b2070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f363c0b2000 tx=0x7f363400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcd977f0df3fcd91 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:09.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363a7fc700 1 -- 172.21.15.3:0/134500485 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400e070 con 0x7f363c0cd7d0 2023-03-03T06:25:09.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363a7fc700 1 -- 172.21.15.3:0/134500485 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363400bb20 con 0x7f363c0cd7d0 2023-03-03T06:25:09.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.816+0000 7f363a7fc700 1 -- 172.21.15.3:0/134500485 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634012430 con 0x7f363c0cd7d0 2023-03-03T06:25:09.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.820+0000 7f3640f23700 1 -- 172.21.15.3:0/134500485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f363c0cd7d0 msgr2=0x7f363c0b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:09.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.820+0000 7f3640f23700 1 --2- 172.21.15.3:0/134500485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f363c0cd7d0 0x7f363c0b2070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f363c0b2000 tx=0x7f363400b330 comp rx=0 tx=0).stop 2023-03-03T06:25:09.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.820+0000 7f3640f23700 1 -- 172.21.15.3:0/134500485 shutdown_connections 2023-03-03T06:25:09.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.820+0000 7f3640f23700 1 --2- 172.21.15.3:0/134500485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f363c0cd7d0 0x7f363c0b2070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.820+0000 7f3640f23700 1 --2- 172.21.15.3:0/134500485 >> v2:172.21.15.3:3300/0 conn(0x7f363c0b4d90 0x7f363c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.820+0000 7f3640f23700 1 -- 172.21.15.3:0/134500485 >> 172.21.15.3:0/134500485 conn(0x7f363c0cf680 msgr2=0x7f363c0cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:09.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.820+0000 7f3640f23700 1 -- 172.21.15.3:0/134500485 shutdown_connections 2023-03-03T06:25:09.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.820+0000 7f3640f23700 1 -- 172.21.15.3:0/134500485 wait complete. 2023-03-03T06:25:09.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.820+0000 7f3640f23700 1 Processor -- start 2023-03-03T06:25:09.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f3640f23700 1 -- start start 2023-03-03T06:25:09.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f3640f23700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f363c0b4d90 0x7f363c154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:09.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f3640f23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f363c0cd7d0 0x7f363c15ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:09.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f3640f23700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f363c15faf0 0x7f363c158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:09.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f3640f23700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c15a740 con 0x7f363c15faf0 2023-03-03T06:25:09.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f3640f23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c159ac0 con 0x7f363c0cd7d0 2023-03-03T06:25:09.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f3640f23700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c159da0 con 0x7f363c0b4d90 2023-03-03T06:25:09.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f363bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f363c15faf0 0x7f363c158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:09.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f363bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f363c15faf0 0x7f363c158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53850/0 (socket says 172.21.15.3:53850) 2023-03-03T06:25:09.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f363bfff700 1 -- 172.21.15.3:0/3776376698 learned_addr learned my addr 172.21.15.3:0/3776376698 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:09.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f363b7fe700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f363c0b4d90 0x7f363c154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:09.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f363affd700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f363c0cd7d0 0x7f363c15ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:09.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f363bfff700 1 -- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f363c0b4d90 msgr2=0x7f363c154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:09.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f363bfff700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f363c0b4d90 0x7f363c154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f363bfff700 1 -- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f363c0cd7d0 msgr2=0x7f363c15ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:09.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f363bfff700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f363c0cd7d0 0x7f363c15ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:09.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f363bfff700 1 -- 172.21.15.3:0/3776376698 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3634009a70 con 0x7f363c15faf0 2023-03-03T06:25:09.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.824+0000 7f363bfff700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f363c15faf0 0x7f363c158f90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3634014040 tx=0x7f3634004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:09.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.828+0000 7f3623fff700 1 -- 172.21.15.3:0/3776376698 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634012430 con 0x7f363c15faf0 2023-03-03T06:25:09.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.828+0000 7f3623fff700 1 -- 172.21.15.3:0/3776376698 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3634020070 con 0x7f363c15faf0 2023-03-03T06:25:09.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.828+0000 7f3623fff700 1 -- 172.21.15.3:0/3776376698 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634004280 con 0x7f363c15faf0 2023-03-03T06:25:09.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.828+0000 7f3640f23700 1 -- 172.21.15.3:0/3776376698 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363c160ea0 con 0x7f363c15faf0 2023-03-03T06:25:09.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.828+0000 7f3640f23700 1 -- 172.21.15.3:0/3776376698 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c160bc0 con 0x7f363c15faf0 2023-03-03T06:25:09.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.828+0000 7f3640f23700 1 -- 172.21.15.3:0/3776376698 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363c0ceef0 con 0x7f363c15faf0 2023-03-03T06:25:09.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.832+0000 7f3623fff700 1 -- 172.21.15.3:0/3776376698 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f363402e070 con 0x7f363c15faf0 2023-03-03T06:25:09.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.832+0000 7f3623fff700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3628064ee0 0x7f3628067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:09.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.832+0000 7f3623fff700 1 -- 172.21.15.3:0/3776376698 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3634022070 con 0x7f363c15faf0 2023-03-03T06:25:09.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.832+0000 7f363b7fe700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3628064ee0 0x7f3628067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:09.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.836+0000 7f363b7fe700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3628064ee0 0x7f3628067390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f362c0011e0 tx=0x7f362c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:09.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:09.836+0000 7f3623fff700 1 -- 172.21.15.3:0/3776376698 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f3634051cc0 con 0x7f363c15faf0 2023-03-03T06:25:10.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.012+0000 7f3640f23700 1 -- 172.21.15.3:0/3776376698 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f363c15a880 con 0x7f363c15faf0 2023-03-03T06:25:10.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.012+0000 7f3623fff700 1 -- 172.21.15.3:0/3776376698 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f363c15a880 con 0x7f363c15faf0 2023-03-03T06:25:10.015 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:25:10.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.012+0000 7f3640f23700 1 -- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3628064ee0 msgr2=0x7f3628067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.012+0000 7f3640f23700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3628064ee0 0x7f3628067390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f362c0011e0 tx=0x7f362c009040 comp rx=0 tx=0).stop 2023-03-03T06:25:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.016+0000 7f3640f23700 1 -- 172.21.15.3:0/3776376698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f363c15faf0 msgr2=0x7f363c158f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.016+0000 7f3640f23700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f363c15faf0 0x7f363c158f90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3634014040 tx=0x7f3634004af0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.016+0000 7f3640f23700 1 -- 172.21.15.3:0/3776376698 shutdown_connections 2023-03-03T06:25:10.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.016+0000 7f3640f23700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3628064ee0 0x7f3628067390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.016+0000 7f3640f23700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f363c15faf0 0x7f363c158f90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.016+0000 7f3640f23700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f363c0cd7d0 0x7f363c15ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.016+0000 7f3640f23700 1 --2- 172.21.15.3:0/3776376698 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f363c0b4d90 0x7f363c154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.016+0000 7f3640f23700 1 -- 172.21.15.3:0/3776376698 >> 172.21.15.3:0/3776376698 conn(0x7f363c0cf680 msgr2=0x7f363c0b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:10.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.016+0000 7f3640f23700 1 -- 172.21.15.3:0/3776376698 shutdown_connections 2023-03-03T06:25:10.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.016+0000 7f3640f23700 1 -- 172.21.15.3:0/3776376698 wait complete. 2023-03-03T06:25:10.051 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:25:10.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.256+0000 7efc0253c700 1 Processor -- start 2023-03-03T06:25:10.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.256+0000 7efc0253c700 1 -- start start 2023-03-03T06:25:10.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc0253c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b4120 0x7efbfc0ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:10.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc0253c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efbfc0b0ed0 0x7efbfc06c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:10.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc0253c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbfc0cb350 con 0x7efbfc0caf00 2023-03-03T06:25:10.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc0253c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbfc0cc200 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc0253c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc0cb580 con 0x7efbfc0b4120 2023-03-03T06:25:10.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc0153a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b4120 0x7efbfc0ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:10.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc0153a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b4120 0x7efbfc0ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:45838/0 (socket says 172.21.15.3:45838) 2023-03-03T06:25:10.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc0153a700 1 -- 172.21.15.3:0/4267885555 learned_addr learned my addr 172.21.15.3:0/4267885555 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:10.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc01d3b700 1 --2- 172.21.15.3:0/4267885555 >> v2:172.21.15.32:3300/0 conn(0x7efbfc0b0ed0 0x7efbfc06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:10.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efbf3fff700 1 -- 172.21.15.3:0/4267885555 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799738068 0 0) 0x7efbfc0cb350 con 0x7efbfc0caf00 2023-03-03T06:25:10.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efbf3fff700 1 -- 172.21.15.3:0/4267885555 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbe80035d0 con 0x7efbfc0caf00 2023-03-03T06:25:10.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc01d3b700 1 -- 172.21.15.3:0/4267885555 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b4120 msgr2=0x7efbfc0ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:10.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc01d3b700 1 --2- 172.21.15.3:0/4267885555 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b4120 0x7efbfc0ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc01d3b700 1 -- 172.21.15.3:0/4267885555 >> v1:172.21.15.32:6789/0 conn(0x7efbfc0caf00 legacy=0x7efbfc0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc01d3b700 1 -- 172.21.15.3:0/4267885555 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc0cb860 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.260+0000 7efc0153a700 1 --2- 172.21.15.3:0/4267885555 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b4120 0x7efbfc0ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:10.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.264+0000 7efc01d3b700 1 --2- 172.21.15.3:0/4267885555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b0ed0 0x7efbfc06c330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009d90 tx=0x7efbf400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7ed91761c1f2e25 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:10.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.264+0000 7efbf3fff700 1 -- 172.21.15.3:0/4267885555 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf400e070 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.264+0000 7efbf3fff700 1 -- 172.21.15.3:0/4267885555 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf400bb20 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.264+0000 7efbf3fff700 1 -- 172.21.15.3:0/4267885555 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4012430 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.264+0000 7efc0253c700 1 -- 172.21.15.3:0/4267885555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b0ed0 msgr2=0x7efbfc06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.264+0000 7efc0253c700 1 --2- 172.21.15.3:0/4267885555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b0ed0 0x7efbfc06c330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009d90 tx=0x7efbf400b330 comp rx=0 tx=0).stop 2023-03-03T06:25:10.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.264+0000 7efc0253c700 1 -- 172.21.15.3:0/4267885555 shutdown_connections 2023-03-03T06:25:10.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.264+0000 7efc0253c700 1 --2- 172.21.15.3:0/4267885555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0b0ed0 0x7efbfc06c330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.264+0000 7efc0253c700 1 --2- 172.21.15.3:0/4267885555 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b4120 0x7efbfc0ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.264+0000 7efc0253c700 1 -- 172.21.15.3:0/4267885555 >> 172.21.15.3:0/4267885555 conn(0x7efbfc0607c0 msgr2=0x7efbfc060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.264+0000 7efc0253c700 1 -- 172.21.15.3:0/4267885555 shutdown_connections 2023-03-03T06:25:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.264+0000 7efc0253c700 1 -- 172.21.15.3:0/4267885555 wait complete. 2023-03-03T06:25:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.268+0000 7efc0253c700 1 Processor -- start 2023-03-03T06:25:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.268+0000 7efc0253c700 1 -- start start 2023-03-03T06:25:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0253c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b0ed0 0x7efbfc0c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0253c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbfc0b4120 0x7efbfc1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:10.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0253c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0caf00 0x7efbfc155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:10.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0253c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc0afd40 con 0x7efbfc0b4120 2023-03-03T06:25:10.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0253c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc0c2240 con 0x7efbfc0caf00 2023-03-03T06:25:10.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0253c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc159420 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0153a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b0ed0 0x7efbfc0c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:10.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc00d39700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbfc0b4120 0x7efbfc1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:10.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0153a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b0ed0 0x7efbfc0c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:45854/0 (socket says 172.21.15.3:45854) 2023-03-03T06:25:10.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0153a700 1 -- 172.21.15.3:0/2145941622 learned_addr learned my addr 172.21.15.3:0/2145941622 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:10.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc01d3b700 1 --2- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0caf00 0x7efbfc155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:10.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0153a700 1 -- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0caf00 msgr2=0x7efbfc155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0153a700 1 --2- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0caf00 0x7efbfc155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0153a700 1 -- 172.21.15.3:0/2145941622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbfc0b4120 msgr2=0x7efbfc1558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0153a700 1 --2- 172.21.15.3:0/2145941622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbfc0b4120 0x7efbfc1558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0153a700 1 -- 172.21.15.3:0/2145941622 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf8009570 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.272+0000 7efc0153a700 1 --2- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b0ed0 0x7efbfc0c7cc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efbf800ec10 tx=0x7efbf800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.276+0000 7efbf1ffb700 1 -- 172.21.15.3:0/2145941622 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf800c850 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.276+0000 7efbf1ffb700 1 -- 172.21.15.3:0/2145941622 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf800c9b0 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.276+0000 7efbf1ffb700 1 -- 172.21.15.3:0/2145941622 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf800cd20 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.276+0000 7efc0253c700 1 -- 172.21.15.3:0/2145941622 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf4009a70 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.276+0000 7efc0253c700 1 -- 172.21.15.3:0/2145941622 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbfc0cbe20 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.276+0000 7efc0253c700 1 -- 172.21.15.3:0/2145941622 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbfc0cbb40 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.280+0000 7efbf1ffb700 1 -- 172.21.15.3:0/2145941622 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7efbf8019030 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.284+0000 7efbf1ffb700 1 --2- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efbe0064ee0 0x7efbe0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:10.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.284+0000 7efbf1ffb700 1 -- 172.21.15.3:0/2145941622 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7efbf8081b60 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.284+0000 7efc00d39700 1 --2- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efbe0064ee0 0x7efbe0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:10.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.284+0000 7efc00d39700 1 --2- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efbe0064ee0 0x7efbe0067390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0ad800 tx=0x7efbec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:10.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.288+0000 7efbf1ffb700 1 -- 172.21.15.3:0/2145941622 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7efbf804ffe0 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.464+0000 7efc0253c700 1 -- 172.21.15.3:0/2145941622 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efbfc0c9070 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.468+0000 7efbf1ffb700 1 -- 172.21.15.3:0/2145941622 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7efbf8053600 con 0x7efbfc0b0ed0 2023-03-03T06:25:10.470 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:25:10.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.472+0000 7efc0253c700 1 -- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efbe0064ee0 msgr2=0x7efbe0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.472+0000 7efc0253c700 1 --2- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efbe0064ee0 0x7efbe0067390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0ad800 tx=0x7efbec009040 comp rx=0 tx=0).stop 2023-03-03T06:25:10.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.472+0000 7efc0253c700 1 -- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b0ed0 msgr2=0x7efbfc0c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.472+0000 7efc0253c700 1 --2- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b0ed0 0x7efbfc0c7cc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efbf800ec10 tx=0x7efbf800ef20 comp rx=0 tx=0).stop 2023-03-03T06:25:10.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.472+0000 7efc0253c700 1 -- 172.21.15.3:0/2145941622 shutdown_connections 2023-03-03T06:25:10.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.472+0000 7efc0253c700 1 --2- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efbe0064ee0 0x7efbe0067390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.472+0000 7efc0253c700 1 --2- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbfc0caf00 0x7efbfc155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.472+0000 7efc0253c700 1 --2- 172.21.15.3:0/2145941622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbfc0b4120 0x7efbfc1558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.472+0000 7efc0253c700 1 --2- 172.21.15.3:0/2145941622 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efbfc0b0ed0 0x7efbfc0c7cc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.472+0000 7efc0253c700 1 -- 172.21.15.3:0/2145941622 >> 172.21.15.3:0/2145941622 conn(0x7efbfc0607c0 msgr2=0x7efbfc0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:10.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.472+0000 7efc0253c700 1 -- 172.21.15.3:0/2145941622 shutdown_connections 2023-03-03T06:25:10.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.472+0000 7efc0253c700 1 -- 172.21.15.3:0/2145941622 wait complete. 2023-03-03T06:25:10.507 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-03-03T06:25:10.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.708+0000 7f83528ab700 1 Processor -- start 2023-03-03T06:25:10.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.712+0000 7f83528ab700 1 -- start start 2023-03-03T06:25:10.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.712+0000 7f83528ab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f834c0c9f60 0x7f834c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:10.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.712+0000 7f83528ab700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f834c0ce100 con 0x7f834c15c510 2023-03-03T06:25:10.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.712+0000 7f83528ab700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f834c0ce450 con 0x7f834c0cd3a0 2023-03-03T06:25:10.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.712+0000 7f83528ab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834c0ce730 con 0x7f834c0c9f60 2023-03-03T06:25:10.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.712+0000 7f83518a9700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f834c15c510 0x7f834c0c9850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45326/0 (socket says 172.21.15.3:45326) 2023-03-03T06:25:10.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.712+0000 7f83518a9700 1 -- 172.21.15.3:0/1387401593 learned_addr learned my addr 172.21.15.3:0/1387401593 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:10.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.712+0000 7f83510a8700 1 --2- 172.21.15.3:0/1387401593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f834c0c9f60 0x7f834c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:10.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.712+0000 7f83508a7700 1 -- 172.21.15.3:0/1387401593 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251058456 0 0) 0x7f834c0ce100 con 0x7f834c15c510 2023-03-03T06:25:10.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83508a7700 1 -- 172.21.15.3:0/1387401593 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83380035d0 con 0x7f834c15c510 2023-03-03T06:25:10.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83508a7700 1 -- 172.21.15.3:0/1387401593 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608613733 0 0) 0x7f834c0ce450 con 0x7f834c0cd3a0 2023-03-03T06:25:10.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83508a7700 1 -- 172.21.15.3:0/1387401593 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f834c0ce100 con 0x7f834c0cd3a0 2023-03-03T06:25:10.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83510a8700 1 -- 172.21.15.3:0/1387401593 >> v1:172.21.15.32:6789/0 conn(0x7f834c0cd3a0 legacy=0x7f834c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83510a8700 1 -- 172.21.15.3:0/1387401593 >> v1:172.21.15.3:6789/0 conn(0x7f834c15c510 legacy=0x7f834c0c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83510a8700 1 -- 172.21.15.3:0/1387401593 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c0cea10 con 0x7f834c0c9f60 2023-03-03T06:25:10.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83510a8700 1 --2- 172.21.15.3:0/1387401593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f834c0c9f60 0x7f834c0cf210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f833c009880 tx=0x7f833c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56193e4dce2f4b76 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:10.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83508a7700 1 -- 172.21.15.3:0/1387401593 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c011830 con 0x7f834c0c9f60 2023-03-03T06:25:10.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83508a7700 1 -- 172.21.15.3:0/1387401593 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833c011990 con 0x7f834c0c9f60 2023-03-03T06:25:10.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83508a7700 1 -- 172.21.15.3:0/1387401593 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c011d20 con 0x7f834c0c9f60 2023-03-03T06:25:10.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83528ab700 1 -- 172.21.15.3:0/1387401593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f834c0c9f60 msgr2=0x7f834c0cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83528ab700 1 --2- 172.21.15.3:0/1387401593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f834c0c9f60 0x7f834c0cf210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f833c009880 tx=0x7f833c009b90 comp rx=0 tx=0).stop 2023-03-03T06:25:10.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83528ab700 1 -- 172.21.15.3:0/1387401593 shutdown_connections 2023-03-03T06:25:10.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83528ab700 1 --2- 172.21.15.3:0/1387401593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f834c0c9f60 0x7f834c0cf210 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.716+0000 7f83528ab700 1 -- 172.21.15.3:0/1387401593 >> 172.21.15.3:0/1387401593 conn(0x7f834c070a30 msgr2=0x7f834c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:10.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.720+0000 7f83528ab700 1 -- 172.21.15.3:0/1387401593 shutdown_connections 2023-03-03T06:25:10.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.720+0000 7f83528ab700 1 -- 172.21.15.3:0/1387401593 wait complete. 2023-03-03T06:25:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.724+0000 7f83528ab700 1 Processor -- start 2023-03-03T06:25:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.724+0000 7f83528ab700 1 -- start start 2023-03-03T06:25:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.724+0000 7f83528ab700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f834c0c9f60 0x7f834c0c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.724+0000 7f83528ab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f834c0cd3a0 0x7f834c14c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.724+0000 7f83528ab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f834c15c510 0x7f834c15b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.724+0000 7f83528ab700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c1597a0 con 0x7f834c15c510 2023-03-03T06:25:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.724+0000 7f83528ab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c0b2210 con 0x7f834c0cd3a0 2023-03-03T06:25:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.724+0000 7f83528ab700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834c0b24f0 con 0x7f834c0c9f60 2023-03-03T06:25:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.724+0000 7f83520aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f834c15c510 0x7f834c15b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.724+0000 7f83518a9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f834c0c9f60 0x7f834c0c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.724+0000 7f83520aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f834c15c510 0x7f834c15b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36018/0 (socket says 172.21.15.3:36018) 2023-03-03T06:25:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.724+0000 7f83520aa700 1 -- 172.21.15.3:0/2489009628 learned_addr learned my addr 172.21.15.3:0/2489009628 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.724+0000 7f83510a8700 1 --2- 172.21.15.3:0/2489009628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f834c0cd3a0 0x7f834c14c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.728+0000 7f83520aa700 1 -- 172.21.15.3:0/2489009628 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f834c0c9f60 msgr2=0x7f834c0c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.728+0000 7f83520aa700 1 --2- 172.21.15.3:0/2489009628 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f834c0c9f60 0x7f834c0c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.728+0000 7f83520aa700 1 -- 172.21.15.3:0/2489009628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f834c0cd3a0 msgr2=0x7f834c14c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.728+0000 7f83520aa700 1 --2- 172.21.15.3:0/2489009628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f834c0cd3a0 0x7f834c14c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.728+0000 7f83520aa700 1 -- 172.21.15.3:0/2489009628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c0094b0 con 0x7f834c15c510 2023-03-03T06:25:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.728+0000 7f83520aa700 1 --2- 172.21.15.3:0/2489009628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f834c15c510 0x7f834c15b120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f834400cc20 tx=0x7f834400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.728+0000 7f83427fc700 1 -- 172.21.15.3:0/2489009628 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8344003c20 con 0x7f834c15c510 2023-03-03T06:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.728+0000 7f83427fc700 1 -- 172.21.15.3:0/2489009628 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8344003d80 con 0x7f834c15c510 2023-03-03T06:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.728+0000 7f83427fc700 1 -- 172.21.15.3:0/2489009628 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834400aae0 con 0x7f834c15c510 2023-03-03T06:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.728+0000 7f83528ab700 1 -- 172.21.15.3:0/2489009628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f834c15c8e0 con 0x7f834c15c510 2023-03-03T06:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.728+0000 7f83528ab700 1 -- 172.21.15.3:0/2489009628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834c0cecf0 con 0x7f834c15c510 2023-03-03T06:25:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.728+0000 7f83528ab700 1 -- 172.21.15.3:0/2489009628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834c0c22f0 con 0x7f834c15c510 2023-03-03T06:25:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.732+0000 7f83427fc700 1 -- 172.21.15.3:0/2489009628 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8344010050 con 0x7f834c15c510 2023-03-03T06:25:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.732+0000 7f83427fc700 1 --2- 172.21.15.3:0/2489009628 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8330064f10 0x7f83300673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.732+0000 7f83427fc700 1 -- 172.21.15.3:0/2489009628 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8344081b90 con 0x7f834c15c510 2023-03-03T06:25:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.736+0000 7f83518a9700 1 --2- 172.21.15.3:0/2489009628 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8330064f10 0x7f83300673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.736+0000 7f83518a9700 1 --2- 172.21.15.3:0/2489009628 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8330064f10 0x7f83300673c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8348007fd0 tx=0x7f8348007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.740+0000 7f83427fc700 1 -- 172.21.15.3:0/2489009628 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f834404ff10 con 0x7f834c15c510 2023-03-03T06:25:10.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.920+0000 7f83528ab700 1 -- 172.21.15.3:0/2489009628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f834c06d500 con 0x7f834c15c510 2023-03-03T06:25:10.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.920+0000 7f83427fc700 1 -- 172.21.15.3:0/2489009628 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8344053530 con 0x7f834c15c510 2023-03-03T06:25:10.924 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:10.924 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-03-03T06:25:10.924 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:25:10.924 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-03-03T06:25:10.924 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-03-03T06:25:10.924 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:25:10.925 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:25:10.925 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2023-03-03T06:25:10.925 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2023-03-03T06:25:10.925 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:25:10.925 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-03-03T06:25:10.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.924+0000 7f83528ab700 1 -- 172.21.15.3:0/2489009628 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8330064f10 msgr2=0x7f83300673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.924+0000 7f83528ab700 1 --2- 172.21.15.3:0/2489009628 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8330064f10 0x7f83300673c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8348007fd0 tx=0x7f8348007de0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.924+0000 7f83528ab700 1 -- 172.21.15.3:0/2489009628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f834c15c510 msgr2=0x7f834c15b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:10.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.924+0000 7f83528ab700 1 --2- 172.21.15.3:0/2489009628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f834c15c510 0x7f834c15b120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f834400cc20 tx=0x7f834400cf30 comp rx=0 tx=0).stop 2023-03-03T06:25:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.924+0000 7f83528ab700 1 -- 172.21.15.3:0/2489009628 shutdown_connections 2023-03-03T06:25:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.924+0000 7f83528ab700 1 --2- 172.21.15.3:0/2489009628 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8330064f10 0x7f83300673c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.924+0000 7f83528ab700 1 --2- 172.21.15.3:0/2489009628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f834c15c510 0x7f834c15b120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.924+0000 7f83528ab700 1 --2- 172.21.15.3:0/2489009628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f834c0cd3a0 0x7f834c14c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.924+0000 7f83528ab700 1 --2- 172.21.15.3:0/2489009628 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f834c0c9f60 0x7f834c0c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.924+0000 7f83528ab700 1 -- 172.21.15.3:0/2489009628 >> 172.21.15.3:0/2489009628 conn(0x7f834c070a30 msgr2=0x7f834c06d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.924+0000 7f83528ab700 1 -- 172.21.15.3:0/2489009628 shutdown_connections 2023-03-03T06:25:10.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:10.924+0000 7f83528ab700 1 -- 172.21.15.3:0/2489009628 wait complete. 2023-03-03T06:25:10.961 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-03-03T06:25:10.962 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-03-03T06:25:11.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.168+0000 7f4d67220700 1 Processor -- start 2023-03-03T06:25:11.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.172+0000 7f4d67220700 1 -- start start 2023-03-03T06:25:11.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d67220700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d60156ba0 0x7f4d60071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:11.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d67220700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d6015d6e0 0x7f4d600cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:11.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d67220700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d600cfd80 0x7f4d600d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:11.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d67220700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d600cf3b0 con 0x7f4d6015d6e0 2023-03-03T06:25:11.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d67220700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d60156f70 con 0x7f4d60156ba0 2023-03-03T06:25:11.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d67220700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d600cd320 con 0x7f4d600cfd80 2023-03-03T06:25:11.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d65a1d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d6015d6e0 0x7f4d600cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:11.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d65a1d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d6015d6e0 0x7f4d600cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36030/0 (socket says 172.21.15.3:36030) 2023-03-03T06:25:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d65a1d700 1 -- 172.21.15.3:0/2875557047 learned_addr learned my addr 172.21.15.3:0/2875557047 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d6621e700 1 --2- 172.21.15.3:0/2875557047 >> v2:172.21.15.32:3300/0 conn(0x7f4d60156ba0 0x7f4d60071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d66a1f700 1 --2- 172.21.15.3:0/2875557047 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d600cfd80 0x7f4d600d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d65a1d700 1 -- 172.21.15.3:0/2875557047 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d600cfd80 msgr2=0x7f4d600d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d65a1d700 1 --2- 172.21.15.3:0/2875557047 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d600cfd80 0x7f4d600d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d65a1d700 1 -- 172.21.15.3:0/2875557047 >> v2:172.21.15.32:3300/0 conn(0x7f4d60156ba0 msgr2=0x7f4d60071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d65a1d700 1 --2- 172.21.15.3:0/2875557047 >> v2:172.21.15.32:3300/0 conn(0x7f4d60156ba0 0x7f4d60071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.176+0000 7f4d65a1d700 1 -- 172.21.15.3:0/2875557047 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d600cd600 con 0x7f4d6015d6e0 2023-03-03T06:25:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.180+0000 7f4d65a1d700 1 --2- 172.21.15.3:0/2875557047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6015d6e0 0x7f4d600cf840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4d60072100 tx=0x7f4d50009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6362c1894396baca server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.180+0000 7f4d6521c700 1 -- 172.21.15.3:0/2875557047 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50011830 con 0x7f4d6015d6e0 2023-03-03T06:25:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.180+0000 7f4d6521c700 1 -- 172.21.15.3:0/2875557047 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d50011990 con 0x7f4d6015d6e0 2023-03-03T06:25:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.180+0000 7f4d6521c700 1 -- 172.21.15.3:0/2875557047 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50011d20 con 0x7f4d6015d6e0 2023-03-03T06:25:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.180+0000 7f4d67220700 1 -- 172.21.15.3:0/2875557047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6015d6e0 msgr2=0x7f4d600cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.180+0000 7f4d67220700 1 --2- 172.21.15.3:0/2875557047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6015d6e0 0x7f4d600cf840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4d60072100 tx=0x7f4d50009bf0 comp rx=0 tx=0).stop 2023-03-03T06:25:11.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.180+0000 7f4d67220700 1 -- 172.21.15.3:0/2875557047 shutdown_connections 2023-03-03T06:25:11.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.180+0000 7f4d67220700 1 --2- 172.21.15.3:0/2875557047 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d600cfd80 0x7f4d600d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:11.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.180+0000 7f4d67220700 1 --2- 172.21.15.3:0/2875557047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6015d6e0 0x7f4d600cf840 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:11.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.180+0000 7f4d67220700 1 --2- 172.21.15.3:0/2875557047 >> v2:172.21.15.32:3300/0 conn(0x7f4d60156ba0 0x7f4d60071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.180+0000 7f4d67220700 1 -- 172.21.15.3:0/2875557047 >> 172.21.15.3:0/2875557047 conn(0x7f4d6006e5c0 msgr2=0x7f4d6006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.180+0000 7f4d67220700 1 -- 172.21.15.3:0/2875557047 shutdown_connections 2023-03-03T06:25:11.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.180+0000 7f4d67220700 1 -- 172.21.15.3:0/2875557047 wait complete. 2023-03-03T06:25:11.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.184+0000 7f4d67220700 1 Processor -- start 2023-03-03T06:25:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.184+0000 7f4d67220700 1 -- start start 2023-03-03T06:25:11.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d67220700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d600cfd80 0x7f4d600b8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:11.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d67220700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d60156ba0 0x7f4d6015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:11.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d67220700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d6015d6e0 0x7f4d60144a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:11.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d67220700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60160730 con 0x7f4d60156ba0 2023-03-03T06:25:11.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d67220700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60160b50 con 0x7f4d6015d6e0 2023-03-03T06:25:11.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d67220700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60151b10 con 0x7f4d600cfd80 2023-03-03T06:25:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d65a1d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d60156ba0 0x7f4d6015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d65a1d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d60156ba0 0x7f4d6015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36032/0 (socket says 172.21.15.3:36032) 2023-03-03T06:25:11.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d65a1d700 1 -- 172.21.15.3:0/3553126539 learned_addr learned my addr 172.21.15.3:0/3553126539 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:11.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d6621e700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d600cfd80 0x7f4d600b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:11.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d66a1f700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d6015d6e0 0x7f4d60144a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:11.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d65a1d700 1 -- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d600cfd80 msgr2=0x7f4d600b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:11.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d65a1d700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d600cfd80 0x7f4d600b8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:11.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d65a1d700 1 -- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d6015d6e0 msgr2=0x7f4d60144a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:11.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d65a1d700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d6015d6e0 0x7f4d60144a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:11.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d65a1d700 1 -- 172.21.15.3:0/3553126539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d50009570 con 0x7f4d60156ba0 2023-03-03T06:25:11.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d6621e700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d600cfd80 0x7f4d600b8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:11.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d65a1d700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d60156ba0 0x7f4d6015f5a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d6006f190 tx=0x7f4d50004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:11.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d56ffd700 1 -- 172.21.15.3:0/3553126539 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50004310 con 0x7f4d60156ba0 2023-03-03T06:25:11.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d56ffd700 1 -- 172.21.15.3:0/3553126539 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5001f070 con 0x7f4d60156ba0 2023-03-03T06:25:11.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d56ffd700 1 -- 172.21.15.3:0/3553126539 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d500036c0 con 0x7f4d60156ba0 2023-03-03T06:25:11.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d67220700 1 -- 172.21.15.3:0/3553126539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d600717c0 con 0x7f4d60156ba0 2023-03-03T06:25:11.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.188+0000 7f4d67220700 1 -- 172.21.15.3:0/3553126539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d600cd8e0 con 0x7f4d60156ba0 2023-03-03T06:25:11.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.192+0000 7f4d67220700 1 -- 172.21.15.3:0/3553126539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d600cef40 con 0x7f4d60156ba0 2023-03-03T06:25:11.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.192+0000 7f4d56ffd700 1 -- 172.21.15.3:0/3553126539 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4d5002a020 con 0x7f4d60156ba0 2023-03-03T06:25:11.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.192+0000 7f4d56ffd700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d44064ee0 0x7f4d44067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:11.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.192+0000 7f4d56ffd700 1 -- 172.21.15.3:0/3553126539 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4d50082fe0 con 0x7f4d60156ba0 2023-03-03T06:25:11.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.196+0000 7f4d6621e700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d44064ee0 0x7f4d44067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:11.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.196+0000 7f4d6621e700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d44064ee0 0x7f4d44067390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009ba0 tx=0x7f4d5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:11.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.200+0000 7f4d56ffd700 1 -- 172.21.15.3:0/3553126539 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4d500513a0 con 0x7f4d60156ba0 2023-03-03T06:25:11.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:11.376+0000 7f4d67220700 1 -- 172.21.15.3:0/3553126539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4d60073e90 con 0x7f4d60156ba0 2023-03-03T06:25:12.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.252+0000 7f4d56ffd700 1 -- 172.21.15.3:0/3553126539 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v21) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4d500150b0 con 0x7f4d60156ba0 2023-03-03T06:25:12.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.260+0000 7f4d67220700 1 -- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d44064ee0 msgr2=0x7f4d44067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:12.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.260+0000 7f4d67220700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d44064ee0 0x7f4d44067390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009ba0 tx=0x7f4d5c008040 comp rx=0 tx=0).stop 2023-03-03T06:25:12.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.260+0000 7f4d67220700 1 -- 172.21.15.3:0/3553126539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d60156ba0 msgr2=0x7f4d6015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:12.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.260+0000 7f4d67220700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d60156ba0 0x7f4d6015f5a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d6006f190 tx=0x7f4d50004770 comp rx=0 tx=0).stop 2023-03-03T06:25:12.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.260+0000 7f4d67220700 1 -- 172.21.15.3:0/3553126539 shutdown_connections 2023-03-03T06:25:12.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.260+0000 7f4d67220700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d44064ee0 0x7f4d44067390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:12.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.260+0000 7f4d67220700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d6015d6e0 0x7f4d60144a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:12.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.260+0000 7f4d67220700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d60156ba0 0x7f4d6015f5a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:12.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.260+0000 7f4d67220700 1 --2- 172.21.15.3:0/3553126539 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d600cfd80 0x7f4d600b8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:12.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.260+0000 7f4d67220700 1 -- 172.21.15.3:0/3553126539 >> 172.21.15.3:0/3553126539 conn(0x7f4d6006e5c0 msgr2=0x7f4d6015ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:12.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.264+0000 7f4d67220700 1 -- 172.21.15.3:0/3553126539 shutdown_connections 2023-03-03T06:25:12.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.264+0000 7f4d67220700 1 -- 172.21.15.3:0/3553126539 wait complete. 2023-03-03T06:25:12.272 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-03-03T06:25:12.302 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-03-03T06:25:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.516+0000 7ff83b234700 1 Processor -- start 2023-03-03T06:25:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.516+0000 7ff83b234700 1 -- start start 2023-03-03T06:25:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.516+0000 7ff83b234700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff8340cf270 0x7ff8340ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.516+0000 7ff83b234700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff834159430 0x7ff8340ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.516+0000 7ff83b234700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8340caef0 con 0x7ff834159060 2023-03-03T06:25:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.516+0000 7ff83b234700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff83415a2c0 con 0x7ff8340cf270 2023-03-03T06:25:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.516+0000 7ff83b234700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff834159920 con 0x7ff834159430 2023-03-03T06:25:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.516+0000 7ff83a232700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff834159060 0x7ff8340ceb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45336/0 (socket says 172.21.15.3:45336) 2023-03-03T06:25:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.516+0000 7ff83a232700 1 -- 172.21.15.3:0/202045607 learned_addr learned my addr 172.21.15.3:0/202045607 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.516+0000 7ff839a31700 1 --2- 172.21.15.3:0/202045607 >> v2:172.21.15.32:3300/0 conn(0x7ff8340cf270 0x7ff8340ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.516+0000 7ff83aa33700 1 --2- 172.21.15.3:0/202045607 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff834159430 0x7ff8340ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:12.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.520+0000 7ff839230700 1 -- 172.21.15.3:0/202045607 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121102501 0 0) 0x7ff8340caef0 con 0x7ff834159060 2023-03-03T06:25:12.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.520+0000 7ff839230700 1 -- 172.21.15.3:0/202045607 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8200035d0 con 0x7ff834159060 2023-03-03T06:25:12.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.520+0000 7ff839a31700 1 -- 172.21.15.3:0/202045607 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff834159430 msgr2=0x7ff8340ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:12.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.520+0000 7ff839a31700 1 --2- 172.21.15.3:0/202045607 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff834159430 0x7ff8340ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.520+0000 7ff839a31700 1 -- 172.21.15.3:0/202045607 >> v1:172.21.15.3:6789/0 conn(0x7ff834159060 legacy=0x7ff8340ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.520+0000 7ff839a31700 1 -- 172.21.15.3:0/202045607 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff834159c00 con 0x7ff8340cf270 2023-03-03T06:25:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.520+0000 7ff839a31700 1 --2- 172.21.15.3:0/202045607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8340cf270 0x7ff8340ca470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff824009850 tx=0x7ff824009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab58b23f6adcc1ce server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.520+0000 7ff839230700 1 -- 172.21.15.3:0/202045607 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824011830 con 0x7ff8340cf270 2023-03-03T06:25:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.520+0000 7ff839230700 1 -- 172.21.15.3:0/202045607 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff824011990 con 0x7ff8340cf270 2023-03-03T06:25:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.520+0000 7ff839230700 1 -- 172.21.15.3:0/202045607 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824011d40 con 0x7ff8340cf270 2023-03-03T06:25:12.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.520+0000 7ff83b234700 1 -- 172.21.15.3:0/202045607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8340cf270 msgr2=0x7ff8340ca470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:12.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.520+0000 7ff83b234700 1 --2- 172.21.15.3:0/202045607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8340cf270 0x7ff8340ca470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff824009850 tx=0x7ff824009b60 comp rx=0 tx=0).stop 2023-03-03T06:25:12.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.524+0000 7ff83b234700 1 -- 172.21.15.3:0/202045607 shutdown_connections 2023-03-03T06:25:12.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.524+0000 7ff83b234700 1 --2- 172.21.15.3:0/202045607 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff834159430 0x7ff8340ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:12.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.524+0000 7ff83b234700 1 --2- 172.21.15.3:0/202045607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8340cf270 0x7ff8340ca470 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:12.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.524+0000 7ff83b234700 1 -- 172.21.15.3:0/202045607 >> 172.21.15.3:0/202045607 conn(0x7ff8340607c0 msgr2=0x7ff834060b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:12.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.524+0000 7ff83b234700 1 -- 172.21.15.3:0/202045607 shutdown_connections 2023-03-03T06:25:12.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.524+0000 7ff83b234700 1 -- 172.21.15.3:0/202045607 wait complete. 2023-03-03T06:25:12.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83b234700 1 Processor -- start 2023-03-03T06:25:12.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83b234700 1 -- start start 2023-03-03T06:25:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83b234700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8340cf270 0x7ff8340c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83b234700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff834159060 0x7ff8341552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83b234700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff834159430 0x7ff834149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83b234700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340c6b70 con 0x7ff834159430 2023-03-03T06:25:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83b234700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff834155830 con 0x7ff8340cf270 2023-03-03T06:25:12.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83b234700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83414a4f0 con 0x7ff834159060 2023-03-03T06:25:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83aa33700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff834159430 0x7ff834149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83aa33700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff834159430 0x7ff834149fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36046/0 (socket says 172.21.15.3:36046) 2023-03-03T06:25:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83aa33700 1 -- 172.21.15.3:0/1521513847 learned_addr learned my addr 172.21.15.3:0/1521513847 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83a232700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8340cf270 0x7ff8340c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff839a31700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff834159060 0x7ff8341552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83aa33700 1 -- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff834159060 msgr2=0x7ff8341552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83aa33700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff834159060 0x7ff8341552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83aa33700 1 -- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8340cf270 msgr2=0x7ff8340c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83aa33700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8340cf270 0x7ff8340c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.528+0000 7ff83aa33700 1 -- 172.21.15.3:0/1521513847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8240094b0 con 0x7ff834159430 2023-03-03T06:25:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.532+0000 7ff83aa33700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff834159430 0x7ff834149fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00efc0 tx=0x7ff82c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.532+0000 7ff82affd700 1 -- 172.21.15.3:0/1521513847 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c00cc90 con 0x7ff834159430 2023-03-03T06:25:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.532+0000 7ff82affd700 1 -- 172.21.15.3:0/1521513847 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c00f040 con 0x7ff834159430 2023-03-03T06:25:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.532+0000 7ff82affd700 1 -- 172.21.15.3:0/1521513847 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0146c0 con 0x7ff834159430 2023-03-03T06:25:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.532+0000 7ff83b234700 1 -- 172.21.15.3:0/1521513847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8340afaf0 con 0x7ff834159430 2023-03-03T06:25:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.532+0000 7ff83b234700 1 -- 172.21.15.3:0/1521513847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff834159ee0 con 0x7ff834159430 2023-03-03T06:25:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.532+0000 7ff83b234700 1 -- 172.21.15.3:0/1521513847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8340afc80 con 0x7ff834159430 2023-03-03T06:25:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.536+0000 7ff82affd700 1 -- 172.21.15.3:0/1521513847 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff82c014820 con 0x7ff834159430 2023-03-03T06:25:12.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.536+0000 7ff82affd700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff818064fe0 0x7ff818067490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:12.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.536+0000 7ff82affd700 1 -- 172.21.15.3:0/1521513847 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5580+0+0 (secure 0 0 0) 0x7ff82c082a30 con 0x7ff834159430 2023-03-03T06:25:12.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.540+0000 7ff83a232700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff818064fe0 0x7ff818067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:12.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.540+0000 7ff83a232700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff818064fe0 0x7ff818067490 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff830007fd0 tx=0x7ff830007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:12.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.544+0000 7ff82affd700 1 -- 172.21.15.3:0/1521513847 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff82c050d40 con 0x7ff834159430 2023-03-03T06:25:12.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:12.724+0000 7ff83b234700 1 -- 172.21.15.3:0/1521513847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7ff8340c6f40 con 0x7ff834159430 2023-03-03T06:25:13.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.272+0000 7ff82affd700 1 -- 172.21.15.3:0/1521513847 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v22) v1 ==== 171+0+0 (secure 0 0 0) 0x7ff8340c6f40 con 0x7ff834159430 2023-03-03T06:25:13.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.288+0000 7ff83b234700 1 -- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff818064fe0 msgr2=0x7ff818067490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:13.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.288+0000 7ff83b234700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff818064fe0 0x7ff818067490 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff830007fd0 tx=0x7ff830007de0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.288+0000 7ff83b234700 1 -- 172.21.15.3:0/1521513847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff834159430 msgr2=0x7ff834149fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:13.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.288+0000 7ff83b234700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff834159430 0x7ff834149fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00efc0 tx=0x7ff82c00c3a0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.296+0000 7ff83b234700 1 -- 172.21.15.3:0/1521513847 shutdown_connections 2023-03-03T06:25:13.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.296+0000 7ff83b234700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff818064fe0 0x7ff818067490 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.296+0000 7ff83b234700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff834159430 0x7ff834149fb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.296+0000 7ff83b234700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff834159060 0x7ff8341552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.296+0000 7ff83b234700 1 --2- 172.21.15.3:0/1521513847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8340cf270 0x7ff8340c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.296+0000 7ff83b234700 1 -- 172.21.15.3:0/1521513847 >> 172.21.15.3:0/1521513847 conn(0x7ff8340607c0 msgr2=0x7ff8340afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:13.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.296+0000 7ff83b234700 1 -- 172.21.15.3:0/1521513847 shutdown_connections 2023-03-03T06:25:13.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.296+0000 7ff83b234700 1 -- 172.21.15.3:0/1521513847 wait complete. 2023-03-03T06:25:13.305 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-03-03T06:25:13.341 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-03-03T06:25:13.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.552+0000 7f9b05df9700 1 Processor -- start 2023-03-03T06:25:13.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.552+0000 7f9b05df9700 1 -- start start 2023-03-03T06:25:13.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.556+0000 7f9b05df9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b00156ba0 0x7f9b00071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:13.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.556+0000 7f9b05df9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b0015d6e0 0x7f9b000cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:13.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.556+0000 7f9b05df9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b000cf3b0 con 0x7f9b000cfd80 2023-03-03T06:25:13.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.556+0000 7f9b05df9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b00156f70 con 0x7f9b0015d6e0 2023-03-03T06:25:13.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.556+0000 7f9b05df9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b000cd320 con 0x7f9b00156ba0 2023-03-03T06:25:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9b055f8700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9b000cfd80 0x7f9b000d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45350/0 (socket says 172.21.15.3:45350) 2023-03-03T06:25:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9b055f8700 1 -- 172.21.15.3:0/2906562918 learned_addr learned my addr 172.21.15.3:0/2906562918 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9b04df7700 1 --2- 172.21.15.3:0/2906562918 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b00156ba0 0x7f9b00071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:13.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9afffff700 1 --2- 172.21.15.3:0/2906562918 >> v2:172.21.15.32:3300/0 conn(0x7f9b0015d6e0 0x7f9b000cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9aff7fe700 1 -- 172.21.15.3:0/2906562918 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410726364 0 0) 0x7f9b000cf3b0 con 0x7f9b000cfd80 2023-03-03T06:25:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9aff7fe700 1 -- 172.21.15.3:0/2906562918 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9af00035d0 con 0x7f9b000cfd80 2023-03-03T06:25:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9b04df7700 1 -- 172.21.15.3:0/2906562918 >> v2:172.21.15.32:3300/0 conn(0x7f9b0015d6e0 msgr2=0x7f9b000cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:13.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9b04df7700 1 --2- 172.21.15.3:0/2906562918 >> v2:172.21.15.32:3300/0 conn(0x7f9b0015d6e0 0x7f9b000cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9b04df7700 1 -- 172.21.15.3:0/2906562918 >> v1:172.21.15.3:6789/0 conn(0x7f9b000cfd80 legacy=0x7f9b000d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:13.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9b04df7700 1 -- 172.21.15.3:0/2906562918 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b000cd600 con 0x7f9b00156ba0 2023-03-03T06:25:13.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9b04df7700 1 --2- 172.21.15.3:0/2906562918 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b00156ba0 0x7f9b00071280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009850 tx=0x7f9af4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2563f41cf4d117b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:13.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9aff7fe700 1 -- 172.21.15.3:0/2906562918 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4011830 con 0x7f9b00156ba0 2023-03-03T06:25:13.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9aff7fe700 1 -- 172.21.15.3:0/2906562918 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af4011990 con 0x7f9b00156ba0 2023-03-03T06:25:13.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.560+0000 7f9aff7fe700 1 -- 172.21.15.3:0/2906562918 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4011d40 con 0x7f9b00156ba0 2023-03-03T06:25:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.564+0000 7f9b05df9700 1 -- 172.21.15.3:0/2906562918 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b00156ba0 msgr2=0x7f9b00071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.564+0000 7f9b05df9700 1 --2- 172.21.15.3:0/2906562918 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b00156ba0 0x7f9b00071280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009850 tx=0x7f9af4009b60 comp rx=0 tx=0).stop 2023-03-03T06:25:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.564+0000 7f9b05df9700 1 -- 172.21.15.3:0/2906562918 shutdown_connections 2023-03-03T06:25:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.564+0000 7f9b05df9700 1 --2- 172.21.15.3:0/2906562918 >> v2:172.21.15.32:3300/0 conn(0x7f9b0015d6e0 0x7f9b000cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.564+0000 7f9b05df9700 1 --2- 172.21.15.3:0/2906562918 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b00156ba0 0x7f9b00071280 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.564+0000 7f9b05df9700 1 -- 172.21.15.3:0/2906562918 >> 172.21.15.3:0/2906562918 conn(0x7f9b0006e5c0 msgr2=0x7f9b0006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.564+0000 7f9b05df9700 1 -- 172.21.15.3:0/2906562918 shutdown_connections 2023-03-03T06:25:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.564+0000 7f9b05df9700 1 -- 172.21.15.3:0/2906562918 wait complete. 2023-03-03T06:25:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.568+0000 7f9b05df9700 1 Processor -- start 2023-03-03T06:25:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.568+0000 7f9b05df9700 1 -- start start 2023-03-03T06:25:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.568+0000 7f9b05df9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b000cfd80 0x7f9b00143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.568+0000 7f9b05df9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b00156ba0 0x7f9b001421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.568+0000 7f9b05df9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b0015d6e0 0x7f9b000c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.568+0000 7f9b05df9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b00143760 con 0x7f9b000cfd80 2023-03-03T06:25:13.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.568+0000 7f9b05df9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b00144130 con 0x7f9b0015d6e0 2023-03-03T06:25:13.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.568+0000 7f9b05df9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b00142b50 con 0x7f9b00156ba0 2023-03-03T06:25:13.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.568+0000 7f9b04df7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b000cfd80 0x7f9b00143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:13.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.568+0000 7f9b04df7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b000cfd80 0x7f9b00143bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36058/0 (socket says 172.21.15.3:36058) 2023-03-03T06:25:13.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.568+0000 7f9b04df7700 1 -- 172.21.15.3:0/3131337560 learned_addr learned my addr 172.21.15.3:0/3131337560 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:13.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.572+0000 7f9b055f8700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b0015d6e0 0x7f9b000c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:13.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.572+0000 7f9afffff700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b00156ba0 0x7f9b001421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:13.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.572+0000 7f9b04df7700 1 -- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b00156ba0 msgr2=0x7f9b001421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:13.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.572+0000 7f9b04df7700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b00156ba0 0x7f9b001421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.572+0000 7f9b04df7700 1 -- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b0015d6e0 msgr2=0x7f9b000c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:13.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.572+0000 7f9b04df7700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b0015d6e0 0x7f9b000c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.572+0000 7f9b04df7700 1 -- 172.21.15.3:0/3131337560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af40094b0 con 0x7f9b000cfd80 2023-03-03T06:25:13.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.572+0000 7f9b04df7700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b000cfd80 0x7f9b00143bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9af40090b0 tx=0x7f9af4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:13.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.572+0000 7f9afd7fa700 1 -- 172.21.15.3:0/3131337560 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4004480 con 0x7f9b000cfd80 2023-03-03T06:25:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.572+0000 7f9afd7fa700 1 -- 172.21.15.3:0/3131337560 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af401f070 con 0x7f9b000cfd80 2023-03-03T06:25:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.572+0000 7f9afd7fa700 1 -- 172.21.15.3:0/3131337560 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4003900 con 0x7f9b000cfd80 2023-03-03T06:25:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.572+0000 7f9b05df9700 1 -- 172.21.15.3:0/3131337560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b000717c0 con 0x7f9b000cfd80 2023-03-03T06:25:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.572+0000 7f9b05df9700 1 -- 172.21.15.3:0/3131337560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b000cd8e0 con 0x7f9b000cfd80 2023-03-03T06:25:13.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.576+0000 7f9b05df9700 1 -- 172.21.15.3:0/3131337560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b000cef40 con 0x7f9b000cfd80 2023-03-03T06:25:13.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.580+0000 7f9afd7fa700 1 -- 172.21.15.3:0/3131337560 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9af402a020 con 0x7f9b000cfd80 2023-03-03T06:25:13.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.580+0000 7f9afd7fa700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9ae4064ee0 0x7f9ae4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:13.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.580+0000 7f9afd7fa700 1 -- 172.21.15.3:0/3131337560 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f9af4083860 con 0x7f9b000cfd80 2023-03-03T06:25:13.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.580+0000 7f9afffff700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9ae4064ee0 0x7f9ae4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:13.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.584+0000 7f9afffff700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9ae4064ee0 0x7f9ae4067390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9b000b8c60 tx=0x7f9aec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:13.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.584+0000 7f9afd7fa700 1 -- 172.21.15.3:0/3131337560 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9af40519f0 con 0x7f9b000cfd80 2023-03-03T06:25:13.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.796+0000 7f9b05df9700 1 -- 172.21.15.3:0/3131337560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9b00073e90 con 0x7f9b000cfd80 2023-03-03T06:25:13.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.812+0000 7f9afd7fa700 1 -- 172.21.15.3:0/3131337560 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9af4006ce0 con 0x7f9b000cfd80 2023-03-03T06:25:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.820+0000 7f9b05df9700 1 -- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9ae4064ee0 msgr2=0x7f9ae4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.820+0000 7f9b05df9700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9ae4064ee0 0x7f9ae4067390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9b000b8c60 tx=0x7f9aec008040 comp rx=0 tx=0).stop 2023-03-03T06:25:13.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.820+0000 7f9b05df9700 1 -- 172.21.15.3:0/3131337560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b000cfd80 msgr2=0x7f9b00143bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.820+0000 7f9b05df9700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b000cfd80 0x7f9b00143bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9af40090b0 tx=0x7f9af4004770 comp rx=0 tx=0).stop 2023-03-03T06:25:13.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.824+0000 7f9b05df9700 1 -- 172.21.15.3:0/3131337560 shutdown_connections 2023-03-03T06:25:13.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.824+0000 7f9b05df9700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9ae4064ee0 0x7f9ae4067390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.824+0000 7f9b05df9700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b0015d6e0 0x7f9b000c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.824+0000 7f9b05df9700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b00156ba0 0x7f9b001421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:13.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.824+0000 7f9b05df9700 1 --2- 172.21.15.3:0/3131337560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b000cfd80 0x7f9b00143bf0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9af40090b0 tx=0x7f9af4004770 comp rx=0 tx=0).stop 2023-03-03T06:25:13.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.824+0000 7f9b05df9700 1 -- 172.21.15.3:0/3131337560 >> 172.21.15.3:0/3131337560 conn(0x7f9b0006e5c0 msgr2=0x7f9b0015ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:13.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.840+0000 7f9b05df9700 1 -- 172.21.15.3:0/3131337560 shutdown_connections 2023-03-03T06:25:13.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:13.840+0000 7f9b05df9700 1 -- 172.21.15.3:0/3131337560 wait complete. 2023-03-03T06:25:13.848 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 3 and data pool 4 2023-03-03T06:25:13.877 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-03-03T06:25:13.877 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-03-03T06:25:14.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.084+0000 7f1f6177c700 1 Processor -- start 2023-03-03T06:25:14.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.084+0000 7f1f6177c700 1 -- start start 2023-03-03T06:25:14.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.088+0000 7f1f6177c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1f5c159160 0x7f1f5c06d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:14.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.088+0000 7f1f6177c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c0cc030 con 0x7f1f5c06ecd0 2023-03-03T06:25:14.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.088+0000 7f1f6177c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c0cbd00 con 0x7f1f5c15d960 2023-03-03T06:25:14.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.088+0000 7f1f6177c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5c06d580 con 0x7f1f5c159160 2023-03-03T06:25:14.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.088+0000 7f1f5affd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1f5c159160 0x7f1f5c06d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:14.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.088+0000 7f1f5bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1f5c06ecd0 0x7f1f5c0cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45366/0 (socket says 172.21.15.3:45366) 2023-03-03T06:25:14.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.088+0000 7f1f5bfff700 1 -- 172.21.15.3:0/1574268584 learned_addr learned my addr 172.21.15.3:0/1574268584 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:14.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.088+0000 7f1f5affd700 1 -- 172.21.15.3:0/1574268584 >> v1:172.21.15.32:6789/0 conn(0x7f1f5c15d960 legacy=0x7f1f5c06e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:14.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.088+0000 7f1f5affd700 1 -- 172.21.15.3:0/1574268584 >> v1:172.21.15.3:6789/0 conn(0x7f1f5c06ecd0 legacy=0x7f1f5c0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:14.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.088+0000 7f1f5affd700 1 -- 172.21.15.3:0/1574268584 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c0cf940 con 0x7f1f5c159160 2023-03-03T06:25:14.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.088+0000 7f1f59ffb700 1 -- 172.21.15.3:0/1574268584 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255953006 0 0) 0x7f1f5c0cc030 con 0x7f1f5c06ecd0 2023-03-03T06:25:14.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.092+0000 7f1f5affd700 1 --2- 172.21.15.3:0/1574268584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f5c159160 0x7f1f5c06d040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009d90 tx=0x7f1f4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295a26271595a9fd server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:14.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.092+0000 7f1f59ffb700 1 -- 172.21.15.3:0/1574268584 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00bdd0 con 0x7f1f5c159160 2023-03-03T06:25:14.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.092+0000 7f1f59ffb700 1 -- 172.21.15.3:0/1574268584 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c011460 con 0x7f1f5c159160 2023-03-03T06:25:14.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.092+0000 7f1f59ffb700 1 -- 172.21.15.3:0/1574268584 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c011810 con 0x7f1f5c159160 2023-03-03T06:25:14.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.092+0000 7f1f6177c700 1 -- 172.21.15.3:0/1574268584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f5c159160 msgr2=0x7f1f5c06d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:14.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.092+0000 7f1f6177c700 1 --2- 172.21.15.3:0/1574268584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f5c159160 0x7f1f5c06d040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009d90 tx=0x7f1f4c00b330 comp rx=0 tx=0).stop 2023-03-03T06:25:14.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.092+0000 7f1f6177c700 1 -- 172.21.15.3:0/1574268584 shutdown_connections 2023-03-03T06:25:14.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.092+0000 7f1f6177c700 1 --2- 172.21.15.3:0/1574268584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f5c159160 0x7f1f5c06d040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:14.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.092+0000 7f1f6177c700 1 -- 172.21.15.3:0/1574268584 >> 172.21.15.3:0/1574268584 conn(0x7f1f5c06ad30 msgr2=0x7f1f5c06b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:14.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.092+0000 7f1f6177c700 1 -- 172.21.15.3:0/1574268584 shutdown_connections 2023-03-03T06:25:14.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.092+0000 7f1f6177c700 1 -- 172.21.15.3:0/1574268584 wait complete. 2023-03-03T06:25:14.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.096+0000 7f1f6177c700 1 Processor -- start 2023-03-03T06:25:14.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.096+0000 7f1f6177c700 1 -- start start 2023-03-03T06:25:14.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.096+0000 7f1f6177c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f5c06ecd0 0x7f1f5c0c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:14.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.096+0000 7f1f6177c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f5c159160 0x7f1f5c0add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:14.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.096+0000 7f1f6177c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f5c15d960 0x7f1f5c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:14.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.096+0000 7f1f6177c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c1f4dc0 con 0x7f1f5c159160 2023-03-03T06:25:14.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.096+0000 7f1f6177c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c1f50a0 con 0x7f1f5c06ecd0 2023-03-03T06:25:14.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.096+0000 7f1f6177c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c1f5380 con 0x7f1f5c15d960 2023-03-03T06:25:14.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.096+0000 7f1f5a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f5c159160 0x7f1f5c0add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:14.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.096+0000 7f1f5a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f5c159160 0x7f1f5c0add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36078/0 (socket says 172.21.15.3:36078) 2023-03-03T06:25:14.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.096+0000 7f1f5a7fc700 1 -- 172.21.15.3:0/169190593 learned_addr learned my addr 172.21.15.3:0/169190593 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:14.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.096+0000 7f1f5affd700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f5c06ecd0 0x7f1f5c0c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:14.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.096+0000 7f1f5bfff700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f5c15d960 0x7f1f5c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:14.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.100+0000 7f1f5a7fc700 1 -- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f5c15d960 msgr2=0x7f1f5c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:14.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.100+0000 7f1f5a7fc700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f5c15d960 0x7f1f5c0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:14.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.100+0000 7f1f5a7fc700 1 -- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f5c06ecd0 msgr2=0x7f1f5c0c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:14.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.100+0000 7f1f5a7fc700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f5c06ecd0 0x7f1f5c0c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:14.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.100+0000 7f1f5a7fc700 1 -- 172.21.15.3:0/169190593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c009a70 con 0x7f1f5c159160 2023-03-03T06:25:14.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.100+0000 7f1f5bfff700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f5c15d960 0x7f1f5c0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:14.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.100+0000 7f1f5a7fc700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f5c159160 0x7f1f5c0add00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000c4d0 tx=0x7f1f5000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:14.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.100+0000 7f1f437fe700 1 -- 172.21.15.3:0/169190593 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f500052d0 con 0x7f1f5c159160 2023-03-03T06:25:14.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.100+0000 7f1f437fe700 1 -- 172.21.15.3:0/169190593 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f50005430 con 0x7f1f5c159160 2023-03-03T06:25:14.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.100+0000 7f1f437fe700 1 -- 172.21.15.3:0/169190593 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f500055a0 con 0x7f1f5c159160 2023-03-03T06:25:14.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.100+0000 7f1f6177c700 1 -- 172.21.15.3:0/169190593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c159530 con 0x7f1f5c159160 2023-03-03T06:25:14.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.100+0000 7f1f6177c700 1 -- 172.21.15.3:0/169190593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c0cfc20 con 0x7f1f5c159160 2023-03-03T06:25:14.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.104+0000 7f1f6177c700 1 -- 172.21.15.3:0/169190593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c0cff00 con 0x7f1f5c159160 2023-03-03T06:25:14.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.104+0000 7f1f437fe700 1 -- 172.21.15.3:0/169190593 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1f50019030 con 0x7f1f5c159160 2023-03-03T06:25:14.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.108+0000 7f1f437fe700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1f48064f10 0x7f1f480673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:14.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.108+0000 7f1f437fe700 1 -- 172.21.15.3:0/169190593 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f50081230 con 0x7f1f5c159160 2023-03-03T06:25:14.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.108+0000 7f1f5affd700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1f48064f10 0x7f1f480673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:14.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.108+0000 7f1f5affd700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1f48064f10 0x7f1f480673c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c000f80 tx=0x7f1f4c00d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:14.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.112+0000 7f1f437fe700 1 -- 172.21.15.3:0/169190593 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1f5004f2e0 con 0x7f1f5c159160 2023-03-03T06:25:14.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.300+0000 7f1f6177c700 1 -- 172.21.15.3:0/169190593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f1f5c04e180 con 0x7f1f5c159160 2023-03-03T06:25:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.812+0000 7f1f437fe700 1 -- 172.21.15.3:0/169190593 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v24) v1 ==== 157+0+0 (secure 0 0 0) 0x7f1f50052900 con 0x7f1f5c159160 2023-03-03T06:25:14.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.820+0000 7f1f6177c700 1 -- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1f48064f10 msgr2=0x7f1f480673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:14.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.820+0000 7f1f6177c700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1f48064f10 0x7f1f480673c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c000f80 tx=0x7f1f4c00d040 comp rx=0 tx=0).stop 2023-03-03T06:25:14.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.820+0000 7f1f6177c700 1 -- 172.21.15.3:0/169190593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f5c159160 msgr2=0x7f1f5c0add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.820+0000 7f1f6177c700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f5c159160 0x7f1f5c0add00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000c4d0 tx=0x7f1f5000c7e0 comp rx=0 tx=0).stop 2023-03-03T06:25:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.820+0000 7f1f6177c700 1 -- 172.21.15.3:0/169190593 shutdown_connections 2023-03-03T06:25:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.820+0000 7f1f6177c700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1f48064f10 0x7f1f480673c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.820+0000 7f1f6177c700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1f5c15d960 0x7f1f5c0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.820+0000 7f1f6177c700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f5c159160 0x7f1f5c0add00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.820+0000 7f1f6177c700 1 --2- 172.21.15.3:0/169190593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f5c06ecd0 0x7f1f5c0c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:14.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.820+0000 7f1f6177c700 1 -- 172.21.15.3:0/169190593 >> 172.21.15.3:0/169190593 conn(0x7f1f5c06ad30 msgr2=0x7f1f5c15ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:14.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.820+0000 7f1f6177c700 1 -- 172.21.15.3:0/169190593 shutdown_connections 2023-03-03T06:25:14.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:14.820+0000 7f1f6177c700 1 -- 172.21.15.3:0/169190593 wait complete. 2023-03-03T06:25:14.858 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-03-03T06:25:15.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.080+0000 7f155d880700 1 Processor -- start 2023-03-03T06:25:15.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.080+0000 7f155d880700 1 -- start start 2023-03-03T06:25:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.080+0000 7f155d880700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f155806da20 0x7f155814f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.080+0000 7f155d880700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15580cafb0 0x7f155815c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.080+0000 7f155d880700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f155814f8c0 con 0x7f15580b1ba0 2023-03-03T06:25:15.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.080+0000 7f155d880700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155815c450 con 0x7f15580cafb0 2023-03-03T06:25:15.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.080+0000 7f155d880700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15580b2a60 con 0x7f155806da20 2023-03-03T06:25:15.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.080+0000 7f154ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15580cafb0 0x7f155815c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:15.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f154ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f15580cafb0 0x7f155815c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36082/0 (socket says 172.21.15.3:36082) 2023-03-03T06:25:15.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f154ffff700 1 -- 172.21.15.3:0/456013470 learned_addr learned my addr 172.21.15.3:0/456013470 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f1556ffd700 1 --2- 172.21.15.3:0/456013470 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f155806da20 0x7f155814f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f154ffff700 1 -- 172.21.15.3:0/456013470 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f155806da20 msgr2=0x7f155814f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:15.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f154ffff700 1 --2- 172.21.15.3:0/456013470 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f155806da20 0x7f155814f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f154ffff700 1 -- 172.21.15.3:0/456013470 >> v1:172.21.15.32:6789/0 conn(0x7f15580b1ba0 legacy=0x7f15580ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f154ffff700 1 -- 172.21.15.3:0/456013470 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155815ce20 con 0x7f15580cafb0 2023-03-03T06:25:15.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f15567fc700 1 -- 172.21.15.3:0/456013470 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144066248 0 0) 0x7f155814f8c0 con 0x7f15580b1ba0 2023-03-03T06:25:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f154ffff700 1 --2- 172.21.15.3:0/456013470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15580cafb0 0x7f155815c8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1540009910 tx=0x7f1540009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee792a2c6f9ebd1d server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:15.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f15567fc700 1 -- 172.21.15.3:0/456013470 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540011830 con 0x7f15580cafb0 2023-03-03T06:25:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f15567fc700 1 -- 172.21.15.3:0/456013470 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1540011990 con 0x7f15580cafb0 2023-03-03T06:25:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f15567fc700 1 -- 172.21.15.3:0/456013470 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540011d40 con 0x7f15580cafb0 2023-03-03T06:25:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f155d880700 1 -- 172.21.15.3:0/456013470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15580cafb0 msgr2=0x7f155815c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f155d880700 1 --2- 172.21.15.3:0/456013470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15580cafb0 0x7f155815c8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1540009910 tx=0x7f1540009c20 comp rx=0 tx=0).stop 2023-03-03T06:25:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f155d880700 1 -- 172.21.15.3:0/456013470 shutdown_connections 2023-03-03T06:25:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f155d880700 1 --2- 172.21.15.3:0/456013470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15580cafb0 0x7f155815c8e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:15.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f155d880700 1 --2- 172.21.15.3:0/456013470 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f155806da20 0x7f155814f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:15.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f155d880700 1 -- 172.21.15.3:0/456013470 >> 172.21.15.3:0/456013470 conn(0x7f155806b4f0 msgr2=0x7f155806b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:15.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f155d880700 1 -- 172.21.15.3:0/456013470 shutdown_connections 2023-03-03T06:25:15.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.084+0000 7f155d880700 1 -- 172.21.15.3:0/456013470 wait complete. 2023-03-03T06:25:15.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.088+0000 7f155d880700 1 Processor -- start 2023-03-03T06:25:15.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.088+0000 7f155d880700 1 -- start start 2023-03-03T06:25:15.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f155d880700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f155806da20 0x7f1558149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f155d880700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15580b1ba0 0x7f15580ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f155d880700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15580cafb0 0x7f15580ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f155d880700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155814c620 con 0x7f155806da20 2023-03-03T06:25:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f155d880700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155814b040 con 0x7f15580b1ba0 2023-03-03T06:25:15.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f155d880700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155814bfd0 con 0x7f15580cafb0 2023-03-03T06:25:15.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f1556ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f155806da20 0x7f1558149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f1556ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f155806da20 0x7f1558149720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36088/0 (socket says 172.21.15.3:36088) 2023-03-03T06:25:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f1556ffd700 1 -- 172.21.15.3:0/2510450911 learned_addr learned my addr 172.21.15.3:0/2510450911 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f154ffff700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15580b1ba0 0x7f15580ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:15.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f1557fff700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15580cafb0 0x7f15580ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:15.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f1556ffd700 1 -- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15580cafb0 msgr2=0x7f15580ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:15.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f1556ffd700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15580cafb0 0x7f15580ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:15.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f1556ffd700 1 -- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15580b1ba0 msgr2=0x7f15580ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:15.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f1556ffd700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15580b1ba0 0x7f15580ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:15.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f1556ffd700 1 -- 172.21.15.3:0/2510450911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1540009570 con 0x7f155806da20 2023-03-03T06:25:15.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f1557fff700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15580cafb0 0x7f15580ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:15.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f154ffff700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15580b1ba0 0x7f15580ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:15.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.092+0000 7f1556ffd700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f155806da20 0x7f1558149720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f154800e9d0 tx=0x7f154800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:15.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.096+0000 7f1554ff9700 1 -- 172.21.15.3:0/2510450911 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548009720 con 0x7f155806da20 2023-03-03T06:25:15.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.096+0000 7f1554ff9700 1 -- 172.21.15.3:0/2510450911 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154800c850 con 0x7f155806da20 2023-03-03T06:25:15.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.096+0000 7f155d880700 1 -- 172.21.15.3:0/2510450911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15580b2780 con 0x7f155806da20 2023-03-03T06:25:15.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.096+0000 7f1554ff9700 1 -- 172.21.15.3:0/2510450911 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154800cbc0 con 0x7f155806da20 2023-03-03T06:25:15.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.096+0000 7f155d880700 1 -- 172.21.15.3:0/2510450911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15580b24a0 con 0x7f155806da20 2023-03-03T06:25:15.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.096+0000 7f155d880700 1 -- 172.21.15.3:0/2510450911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155815c0e0 con 0x7f155806da20 2023-03-03T06:25:15.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.096+0000 7f1554ff9700 1 -- 172.21.15.3:0/2510450911 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1548019030 con 0x7f155806da20 2023-03-03T06:25:15.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.100+0000 7f1554ff9700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1538064ee0 0x7f1538067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:15.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.100+0000 7f1554ff9700 1 -- 172.21.15.3:0/2510450911 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f15480820a0 con 0x7f155806da20 2023-03-03T06:25:15.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.100+0000 7f154ffff700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1538064ee0 0x7f1538067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:15.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.104+0000 7f154ffff700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1538064ee0 0x7f1538067390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1540009910 tx=0x7f1540008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:15.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.108+0000 7f1554ff9700 1 -- 172.21.15.3:0/2510450911 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1548050010 con 0x7f155806da20 2023-03-03T06:25:15.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:15.288+0000 7f155d880700 1 -- 172.21.15.3:0/2510450911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f155814be30 con 0x7f155806da20 2023-03-03T06:25:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.320+0000 7f1554ff9700 1 -- 172.21.15.3:0/2510450911 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v26) v1 ==== 228+0+0 (secure 0 0 0) 0x7f155814be30 con 0x7f155806da20 2023-03-03T06:25:17.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.336+0000 7f155d880700 1 -- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1538064ee0 msgr2=0x7f1538067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:17.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.336+0000 7f155d880700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1538064ee0 0x7f1538067390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1540009910 tx=0x7f1540008030 comp rx=0 tx=0).stop 2023-03-03T06:25:17.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.336+0000 7f155d880700 1 -- 172.21.15.3:0/2510450911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f155806da20 msgr2=0x7f1558149720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:17.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.336+0000 7f155d880700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f155806da20 0x7f1558149720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f154800e9d0 tx=0x7f154800ed90 comp rx=0 tx=0).stop 2023-03-03T06:25:17.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.340+0000 7f155d880700 1 -- 172.21.15.3:0/2510450911 shutdown_connections 2023-03-03T06:25:17.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.340+0000 7f155d880700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1538064ee0 0x7f1538067390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:17.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.340+0000 7f155d880700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f15580cafb0 0x7f15580ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:17.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.340+0000 7f155d880700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15580b1ba0 0x7f15580ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:17.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.340+0000 7f155d880700 1 --2- 172.21.15.3:0/2510450911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f155806da20 0x7f1558149720 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:17.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.340+0000 7f155d880700 1 -- 172.21.15.3:0/2510450911 >> 172.21.15.3:0/2510450911 conn(0x7f155806b4f0 msgr2=0x7f155814e620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:17.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.340+0000 7f155d880700 1 -- 172.21.15.3:0/2510450911 shutdown_connections 2023-03-03T06:25:17.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.340+0000 7f155d880700 1 -- 172.21.15.3:0/2510450911 wait complete. 2023-03-03T06:25:17.345 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data_ec' created 2023-03-03T06:25:17.373 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-03-03T06:25:17.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.584+0000 7f9e5f59e700 1 Processor -- start 2023-03-03T06:25:17.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.588+0000 7f9e5f59e700 1 -- start start 2023-03-03T06:25:17.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.588+0000 7f9e5f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e6015c370 0x7f9e6006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:17.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.588+0000 7f9e5f59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e600ccbc0 con 0x7f9e600cb520 2023-03-03T06:25:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.588+0000 7f9e5f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e600cbae0 con 0x7f9e600b0ed0 2023-03-03T06:25:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.588+0000 7f9e5f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e600ce520 con 0x7f9e6015c370 2023-03-03T06:25:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.588+0000 7f9e55d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9e600cb520 0x7f9e600b07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45382/0 (socket says 172.21.15.3:45382) 2023-03-03T06:25:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.588+0000 7f9e55d9b700 1 -- 172.21.15.3:0/2721901185 learned_addr learned my addr 172.21.15.3:0/2721901185 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:17.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.588+0000 7f9e5e59c700 1 --2- 172.21.15.3:0/2721901185 >> v2:172.21.15.32:3300/0 conn(0x7f9e6015c370 0x7f9e6006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:17.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5dd9b700 1 -- 172.21.15.3:0/2721901185 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258515476 0 0) 0x7f9e600ccbc0 con 0x7f9e600cb520 2023-03-03T06:25:17.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5dd9b700 1 -- 172.21.15.3:0/2721901185 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e400035d0 con 0x7f9e600cb520 2023-03-03T06:25:17.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5dd9b700 1 -- 172.21.15.3:0/2721901185 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231446114 0 0) 0x7f9e600cbae0 con 0x7f9e600b0ed0 2023-03-03T06:25:17.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5e59c700 1 -- 172.21.15.3:0/2721901185 >> v1:172.21.15.32:6789/0 conn(0x7f9e600b0ed0 legacy=0x7f9e601601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:17.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5e59c700 1 -- 172.21.15.3:0/2721901185 >> v1:172.21.15.3:6789/0 conn(0x7f9e600cb520 legacy=0x7f9e600b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:17.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5e59c700 1 -- 172.21.15.3:0/2721901185 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e600cc500 con 0x7f9e6015c370 2023-03-03T06:25:17.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5e59c700 1 --2- 172.21.15.3:0/2721901185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e6015c370 0x7f9e6006c090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009850 tx=0x7f9e50009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c03ed272b7dcc248 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:17.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5dd9b700 1 -- 172.21.15.3:0/2721901185 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50011830 con 0x7f9e6015c370 2023-03-03T06:25:17.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5dd9b700 1 -- 172.21.15.3:0/2721901185 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50011990 con 0x7f9e6015c370 2023-03-03T06:25:17.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5dd9b700 1 -- 172.21.15.3:0/2721901185 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50011d40 con 0x7f9e6015c370 2023-03-03T06:25:17.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5f59e700 1 -- 172.21.15.3:0/2721901185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e6015c370 msgr2=0x7f9e6006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:17.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5f59e700 1 --2- 172.21.15.3:0/2721901185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e6015c370 0x7f9e6006c090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009850 tx=0x7f9e50009b60 comp rx=0 tx=0).stop 2023-03-03T06:25:17.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5f59e700 1 -- 172.21.15.3:0/2721901185 shutdown_connections 2023-03-03T06:25:17.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5f59e700 1 --2- 172.21.15.3:0/2721901185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e6015c370 0x7f9e6006c090 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:17.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5f59e700 1 -- 172.21.15.3:0/2721901185 >> 172.21.15.3:0/2721901185 conn(0x7f9e600607c0 msgr2=0x7f9e60060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:17.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5f59e700 1 -- 172.21.15.3:0/2721901185 shutdown_connections 2023-03-03T06:25:17.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.592+0000 7f9e5f59e700 1 -- 172.21.15.3:0/2721901185 wait complete. 2023-03-03T06:25:17.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.596+0000 7f9e5f59e700 1 Processor -- start 2023-03-03T06:25:17.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.596+0000 7f9e5f59e700 1 -- start start 2023-03-03T06:25:17.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e600b0ed0 0x7f9e601f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:17.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e600cb520 0x7f9e601f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:17.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e6015c370 0x7f9e600b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:17.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e601f5d50 con 0x7f9e600b0ed0 2023-03-03T06:25:17.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e600b1840 con 0x7f9e600cb520 2023-03-03T06:25:17.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5f59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e600b1b20 con 0x7f9e6015c370 2023-03-03T06:25:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e600b0ed0 0x7f9e601f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e600b0ed0 0x7f9e601f5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36094/0 (socket says 172.21.15.3:36094) 2023-03-03T06:25:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5e59c700 1 -- 172.21.15.3:0/3180307651 learned_addr learned my addr 172.21.15.3:0/3180307651 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:17.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e55d9b700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e600cb520 0x7f9e601f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5ed9d700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e6015c370 0x7f9e600b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5e59c700 1 -- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e6015c370 msgr2=0x7f9e600b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5e59c700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e6015c370 0x7f9e600b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5e59c700 1 -- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e600cb520 msgr2=0x7f9e601f68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5e59c700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e600cb520 0x7f9e601f68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5e59c700 1 -- 172.21.15.3:0/3180307651 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e500094b0 con 0x7f9e600b0ed0 2023-03-03T06:25:17.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5ed9d700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e6015c370 0x7f9e600b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e55d9b700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e600cb520 0x7f9e601f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.600+0000 7f9e5e59c700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e600b0ed0 0x7f9e601f5810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9e500090b0 tx=0x7f9e50004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.604+0000 7f9e577fe700 1 -- 172.21.15.3:0/3180307651 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50019b60 con 0x7f9e600b0ed0 2023-03-03T06:25:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.604+0000 7f9e577fe700 1 -- 172.21.15.3:0/3180307651 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5001e070 con 0x7f9e600b0ed0 2023-03-03T06:25:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.604+0000 7f9e5f59e700 1 -- 172.21.15.3:0/3180307651 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6015c740 con 0x7f9e600b0ed0 2023-03-03T06:25:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.604+0000 7f9e577fe700 1 -- 172.21.15.3:0/3180307651 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5001a420 con 0x7f9e600b0ed0 2023-03-03T06:25:17.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.604+0000 7f9e5f59e700 1 -- 172.21.15.3:0/3180307651 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e600cc7e0 con 0x7f9e600b0ed0 2023-03-03T06:25:17.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.604+0000 7f9e5f59e700 1 -- 172.21.15.3:0/3180307651 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6015d400 con 0x7f9e600b0ed0 2023-03-03T06:25:17.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.608+0000 7f9e577fe700 1 -- 172.21.15.3:0/3180307651 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9e5000f050 con 0x7f9e600b0ed0 2023-03-03T06:25:17.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.608+0000 7f9e577fe700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9e44064ee0 0x7f9e44067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:17.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.608+0000 7f9e577fe700 1 -- 172.21.15.3:0/3180307651 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f9e500832c0 con 0x7f9e600b0ed0 2023-03-03T06:25:17.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.612+0000 7f9e55d9b700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9e44064ee0 0x7f9e44067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:17.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.612+0000 7f9e55d9b700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9e44064ee0 0x7f9e44067390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e60154550 tx=0x7f9e48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:17.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.616+0000 7f9e577fe700 1 -- 172.21.15.3:0/3180307651 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9e500510a0 con 0x7f9e600b0ed0 2023-03-03T06:25:17.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:17.796+0000 7f9e5f59e700 1 -- 172.21.15.3:0/3180307651 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f9e601e4ee0 con 0x7f9e600b0ed0 2023-03-03T06:25:18.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.332+0000 7f9e577fe700 1 -- 172.21.15.3:0/3180307651 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9e500546c0 con 0x7f9e600b0ed0 2023-03-03T06:25:18.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.344+0000 7f9e5f59e700 1 -- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9e44064ee0 msgr2=0x7f9e44067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:18.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.344+0000 7f9e5f59e700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9e44064ee0 0x7f9e44067390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e60154550 tx=0x7f9e48009040 comp rx=0 tx=0).stop 2023-03-03T06:25:18.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.344+0000 7f9e5f59e700 1 -- 172.21.15.3:0/3180307651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e600b0ed0 msgr2=0x7f9e601f5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:18.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.344+0000 7f9e5f59e700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e600b0ed0 0x7f9e601f5810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9e500090b0 tx=0x7f9e50004770 comp rx=0 tx=0).stop 2023-03-03T06:25:18.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.356+0000 7f9e5f59e700 1 -- 172.21.15.3:0/3180307651 shutdown_connections 2023-03-03T06:25:18.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.356+0000 7f9e5f59e700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9e44064ee0 0x7f9e44067390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.356+0000 7f9e5f59e700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e6015c370 0x7f9e600b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.356+0000 7f9e5f59e700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e600cb520 0x7f9e601f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.356+0000 7f9e5f59e700 1 --2- 172.21.15.3:0/3180307651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e600b0ed0 0x7f9e601f5810 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.356+0000 7f9e5f59e700 1 -- 172.21.15.3:0/3180307651 >> 172.21.15.3:0/3180307651 conn(0x7f9e600607c0 msgr2=0x7f9e600c4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:18.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.360+0000 7f9e5f59e700 1 -- 172.21.15.3:0/3180307651 shutdown_connections 2023-03-03T06:25:18.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.360+0000 7f9e5f59e700 1 -- 172.21.15.3:0/3180307651 wait complete. 2023-03-03T06:25:18.371 INFO:teuthology.orchestra.run.smithi003.stderr:set pool 5 allow_ec_overwrites to true 2023-03-03T06:25:18.405 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-03-03T06:25:18.428 INFO:tasks.ceph.mds.b.smithi032.stderr:2023-03-03T06:25:18.426+0000 7f60b0589700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-03-03T06:25:18.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.612+0000 7f493759e700 1 Processor -- start 2023-03-03T06:25:18.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.612+0000 7f493759e700 1 -- start start 2023-03-03T06:25:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.612+0000 7f493759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f493806a780 0x7f49380cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:18.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.612+0000 7f493759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f49380cb9a0 0x7f4938158400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.616+0000 7f493759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493815c230 con 0x7f4938158940 2023-03-03T06:25:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.616+0000 7f493759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49380caac0 con 0x7f493806a780 2023-03-03T06:25:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.616+0000 7f493759e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49380cd130 con 0x7f49380cb9a0 2023-03-03T06:25:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.616+0000 7f4935d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f49380cb9a0 0x7f4938158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.616+0000 7f493659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f493806a780 0x7f49380cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.616+0000 7f4935d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f49380cb9a0 0x7f4938158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:45986/0 (socket says 172.21.15.3:45986) 2023-03-03T06:25:18.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.616+0000 7f4935d9b700 1 -- 172.21.15.3:0/4149197240 learned_addr learned my addr 172.21.15.3:0/4149197240 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.616+0000 7f4935d9b700 1 -- 172.21.15.3:0/4149197240 >> v2:172.21.15.32:3300/0 conn(0x7f493806a780 msgr2=0x7f49380cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.616+0000 7f4935d9b700 1 --2- 172.21.15.3:0/4149197240 >> v2:172.21.15.32:3300/0 conn(0x7f493806a780 0x7f49380cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.616+0000 7f4935d9b700 1 -- 172.21.15.3:0/4149197240 >> v1:172.21.15.32:6789/0 conn(0x7f4938158940 legacy=0x7f49380709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:18.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.616+0000 7f4935d9b700 1 -- 172.21.15.3:0/4149197240 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49380cbf60 con 0x7f49380cb9a0 2023-03-03T06:25:18.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.616+0000 7f4935d9b700 1 --2- 172.21.15.3:0/4149197240 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f49380cb9a0 0x7f4938158400 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f492c00bbb0 tx=0x7f492c00bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28a6ab3509ca4951 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:18.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.620+0000 7f493559a700 1 -- 172.21.15.3:0/4149197240 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c00dd30 con 0x7f49380cb9a0 2023-03-03T06:25:18.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.620+0000 7f493559a700 1 -- 172.21.15.3:0/4149197240 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c00de90 con 0x7f49380cb9a0 2023-03-03T06:25:18.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.620+0000 7f493559a700 1 -- 172.21.15.3:0/4149197240 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c010690 con 0x7f49380cb9a0 2023-03-03T06:25:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.620+0000 7f493759e700 1 -- 172.21.15.3:0/4149197240 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f49380cb9a0 msgr2=0x7f4938158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:18.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.620+0000 7f493759e700 1 --2- 172.21.15.3:0/4149197240 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f49380cb9a0 0x7f4938158400 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f492c00bbb0 tx=0x7f492c00bec0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.620+0000 7f493759e700 1 -- 172.21.15.3:0/4149197240 shutdown_connections 2023-03-03T06:25:18.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.620+0000 7f493759e700 1 --2- 172.21.15.3:0/4149197240 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f49380cb9a0 0x7f4938158400 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.620+0000 7f493759e700 1 --2- 172.21.15.3:0/4149197240 >> v2:172.21.15.32:3300/0 conn(0x7f493806a780 0x7f49380cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.620+0000 7f493759e700 1 -- 172.21.15.3:0/4149197240 >> 172.21.15.3:0/4149197240 conn(0x7f49380607c0 msgr2=0x7f4938060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.620+0000 7f493759e700 1 -- 172.21.15.3:0/4149197240 shutdown_connections 2023-03-03T06:25:18.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.620+0000 7f493759e700 1 -- 172.21.15.3:0/4149197240 wait complete. 2023-03-03T06:25:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.624+0000 7f493759e700 1 Processor -- start 2023-03-03T06:25:18.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.624+0000 7f493759e700 1 -- start start 2023-03-03T06:25:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f493759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f493806a780 0x7f4938159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:18.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f493759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49380cb9a0 0x7f4938154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:18.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f493759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4938158940 0x7f49381e4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:18.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f493759e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4938159750 con 0x7f4938158940 2023-03-03T06:25:18.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f493759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493815b1f0 con 0x7f49380cb9a0 2023-03-03T06:25:18.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f493759e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4938158630 con 0x7f493806a780 2023-03-03T06:25:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f4936d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4938158940 0x7f49381e4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:18.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f493659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f493806a780 0x7f4938159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:18.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f4936d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4938158940 0x7f49381e4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36100/0 (socket says 172.21.15.3:36100) 2023-03-03T06:25:18.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f4936d9d700 1 -- 172.21.15.3:0/625470627 learned_addr learned my addr 172.21.15.3:0/625470627 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:18.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f4935d9b700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49380cb9a0 0x7f4938154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:18.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f493659c700 1 -- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49380cb9a0 msgr2=0x7f4938154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:18.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f493659c700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49380cb9a0 0x7f4938154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f493659c700 1 -- 172.21.15.3:0/625470627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4938158940 msgr2=0x7f49381e4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:18.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f493659c700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4938158940 0x7f49381e4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f493659c700 1 -- 172.21.15.3:0/625470627 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c00b810 con 0x7f493806a780 2023-03-03T06:25:18.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f4936d9d700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4938158940 0x7f49381e4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:18.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f4935d9b700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49380cb9a0 0x7f4938154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:18.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.628+0000 7f493659c700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f493806a780 0x7f4938159b70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f492800ebc0 tx=0x7f492800eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:18.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.632+0000 7f4926ffd700 1 -- 172.21.15.3:0/625470627 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800ca80 con 0x7f493806a780 2023-03-03T06:25:18.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.632+0000 7f4926ffd700 1 -- 172.21.15.3:0/625470627 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492800f040 con 0x7f493806a780 2023-03-03T06:25:18.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.632+0000 7f493759e700 1 -- 172.21.15.3:0/625470627 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493815c7d0 con 0x7f493806a780 2023-03-03T06:25:18.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.632+0000 7f4926ffd700 1 -- 172.21.15.3:0/625470627 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800ce20 con 0x7f493806a780 2023-03-03T06:25:18.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.632+0000 7f493759e700 1 -- 172.21.15.3:0/625470627 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493815c4f0 con 0x7f493806a780 2023-03-03T06:25:18.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.632+0000 7f493759e700 1 -- 172.21.15.3:0/625470627 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493815cab0 con 0x7f493806a780 2023-03-03T06:25:18.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.636+0000 7f4926ffd700 1 -- 172.21.15.3:0/625470627 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4928014480 con 0x7f493806a780 2023-03-03T06:25:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.636+0000 7f4926ffd700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4918064ee0 0x7f4918067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.636+0000 7f4926ffd700 1 -- 172.21.15.3:0/625470627 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f4928082d90 con 0x7f493806a780 2023-03-03T06:25:18.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.636+0000 7f4935d9b700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4918064ee0 0x7f4918067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.640+0000 7f4935d9b700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4918064ee0 0x7f4918067390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f492c00bfd0 tx=0x7f492c01b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.644+0000 7f4926ffd700 1 -- 172.21.15.3:0/625470627 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4928050b10 con 0x7f493806a780 2023-03-03T06:25:18.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.860+0000 7f493759e700 1 -- 172.21.15.3:0/625470627 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f4938073e90 con 0x7f493806a780 2023-03-03T06:25:18.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.876+0000 7f4926ffd700 1 -- 172.21.15.3:0/625470627 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7f4928054130 con 0x7f493806a780 2023-03-03T06:25:18.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.880+0000 7f493759e700 1 -- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4918064ee0 msgr2=0x7f4918067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:18.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.880+0000 7f493759e700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4918064ee0 0x7f4918067390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f492c00bfd0 tx=0x7f492c01b040 comp rx=0 tx=0).stop 2023-03-03T06:25:18.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.880+0000 7f493759e700 1 -- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f493806a780 msgr2=0x7f4938159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:18.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.880+0000 7f493759e700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f493806a780 0x7f4938159b70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f492800ebc0 tx=0x7f492800eed0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.880+0000 7f493759e700 1 -- 172.21.15.3:0/625470627 shutdown_connections 2023-03-03T06:25:18.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.880+0000 7f493759e700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4918064ee0 0x7f4918067390 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.880+0000 7f493759e700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4938158940 0x7f49381e4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.880+0000 7f493759e700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49380cb9a0 0x7f4938154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.880+0000 7f493759e700 1 --2- 172.21.15.3:0/625470627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f493806a780 0x7f4938159b70 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:18.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.880+0000 7f493759e700 1 -- 172.21.15.3:0/625470627 >> 172.21.15.3:0/625470627 conn(0x7f49380607c0 msgr2=0x7f49380c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:18.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.880+0000 7f493759e700 1 -- 172.21.15.3:0/625470627 shutdown_connections 2023-03-03T06:25:18.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:18.880+0000 7f493759e700 1 -- 172.21.15.3:0/625470627 wait complete. 2023-03-03T06:25:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:added data pool 5 to fsmap 2023-03-03T06:25:18.916 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-03T06:25:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.124+0000 7f406c936700 1 Processor -- start 2023-03-03T06:25:19.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.128+0000 7f406c936700 1 -- start start 2023-03-03T06:25:19.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.128+0000 7f406c936700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4068159120 0x7f40680aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.128+0000 7f406c936700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40680ce890 0x7f40680cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.128+0000 7f406c936700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40680cc590 con 0x7f406815d6c0 2023-03-03T06:25:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.128+0000 7f406c936700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406815db10 con 0x7f40680ce890 2023-03-03T06:25:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.128+0000 7f406c936700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40681594f0 con 0x7f4068159120 2023-03-03T06:25:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.132+0000 7f4066d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4068159120 0x7f40680aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.132+0000 7f406759e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40680ce890 0x7f40680cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:19.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.132+0000 7f405e59c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f406815d6c0 0x7f40680ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:57758/0 (socket says 172.21.15.3:57758) 2023-03-03T06:25:19.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.132+0000 7f405e59c700 1 -- 172.21.15.3:0/3008957433 learned_addr learned my addr 172.21.15.3:0/3008957433 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:19.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.132+0000 7f4066d9d700 1 -- 172.21.15.3:0/3008957433 >> v2:172.21.15.3:3300/0 conn(0x7f40680ce890 msgr2=0x7f40680cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:19.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.132+0000 7f4066d9d700 1 --2- 172.21.15.3:0/3008957433 >> v2:172.21.15.3:3300/0 conn(0x7f40680ce890 0x7f40680cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.132+0000 7f4066d9d700 1 -- 172.21.15.3:0/3008957433 >> v1:172.21.15.32:6789/0 conn(0x7f406815d6c0 legacy=0x7f40680ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:19.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.132+0000 7f4066d9d700 1 -- 172.21.15.3:0/3008957433 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40680cc7d0 con 0x7f4068159120 2023-03-03T06:25:19.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.132+0000 7f4066d9d700 1 --2- 172.21.15.3:0/3008957433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4068159120 0x7f40680aed10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f406806b190 tx=0x7f4058009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d99db77c9c838b4 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:19.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.132+0000 7f406659c700 1 -- 172.21.15.3:0/3008957433 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4058011830 con 0x7f4068159120 2023-03-03T06:25:19.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.132+0000 7f406659c700 1 -- 172.21.15.3:0/3008957433 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4058011990 con 0x7f4068159120 2023-03-03T06:25:19.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.132+0000 7f406659c700 1 -- 172.21.15.3:0/3008957433 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4058011d20 con 0x7f4068159120 2023-03-03T06:25:19.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.136+0000 7f406c936700 1 -- 172.21.15.3:0/3008957433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4068159120 msgr2=0x7f40680aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:19.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.136+0000 7f406c936700 1 --2- 172.21.15.3:0/3008957433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4068159120 0x7f40680aed10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f406806b190 tx=0x7f4058009b30 comp rx=0 tx=0).stop 2023-03-03T06:25:19.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.136+0000 7f406c936700 1 -- 172.21.15.3:0/3008957433 shutdown_connections 2023-03-03T06:25:19.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.136+0000 7f406c936700 1 --2- 172.21.15.3:0/3008957433 >> v2:172.21.15.3:3300/0 conn(0x7f40680ce890 0x7f40680cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.136+0000 7f406c936700 1 --2- 172.21.15.3:0/3008957433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4068159120 0x7f40680aed10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.136+0000 7f406c936700 1 -- 172.21.15.3:0/3008957433 >> 172.21.15.3:0/3008957433 conn(0x7f4068069d70 msgr2=0x7f406806a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:19.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.136+0000 7f406c936700 1 -- 172.21.15.3:0/3008957433 shutdown_connections 2023-03-03T06:25:19.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.136+0000 7f406c936700 1 -- 172.21.15.3:0/3008957433 wait complete. 2023-03-03T06:25:19.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.136+0000 7f406c936700 1 Processor -- start 2023-03-03T06:25:19.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406c936700 1 -- start start 2023-03-03T06:25:19.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406c936700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40680ce890 0x7f40680c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:19.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406c936700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4068159120 0x7f40680c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:19.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406c936700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406815d6c0 0x7f4068146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:19.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406c936700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40680c4e20 con 0x7f406815d6c0 2023-03-03T06:25:19.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406c936700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40680c5840 con 0x7f4068159120 2023-03-03T06:25:19.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406c936700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40680c5560 con 0x7f40680ce890 2023-03-03T06:25:19.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406815d6c0 0x7f4068146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:19.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406815d6c0 0x7f4068146ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36118/0 (socket says 172.21.15.3:36118) 2023-03-03T06:25:19.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406759e700 1 -- 172.21.15.3:0/3314469113 learned_addr learned my addr 172.21.15.3:0/3314469113 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:19.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f405e59c700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4068159120 0x7f40680c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:19.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f4066d9d700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40680ce890 0x7f40680c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:19.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406759e700 1 -- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40680ce890 msgr2=0x7f40680c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:19.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406759e700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40680ce890 0x7f40680c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406759e700 1 -- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4068159120 msgr2=0x7f40680c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:19.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406759e700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4068159120 0x7f40680c48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.140+0000 7f406759e700 1 -- 172.21.15.3:0/3314469113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40580094b0 con 0x7f406815d6c0 2023-03-03T06:25:19.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.144+0000 7f406759e700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406815d6c0 0x7f4068146ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f406000efc0 tx=0x7f406000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:19.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.144+0000 7f405ffff700 1 -- 172.21.15.3:0/3314469113 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406000cad0 con 0x7f406815d6c0 2023-03-03T06:25:19.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.144+0000 7f405ffff700 1 -- 172.21.15.3:0/3314469113 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406000f040 con 0x7f406815d6c0 2023-03-03T06:25:19.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.144+0000 7f406c936700 1 -- 172.21.15.3:0/3314469113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406815cbe0 con 0x7f406815d6c0 2023-03-03T06:25:19.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.144+0000 7f405ffff700 1 -- 172.21.15.3:0/3314469113 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406000ce70 con 0x7f406815d6c0 2023-03-03T06:25:19.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.144+0000 7f406c936700 1 -- 172.21.15.3:0/3314469113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40680ccab0 con 0x7f406815d6c0 2023-03-03T06:25:19.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.144+0000 7f406c936700 1 -- 172.21.15.3:0/3314469113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406815cec0 con 0x7f406815d6c0 2023-03-03T06:25:19.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.148+0000 7f405ffff700 1 -- 172.21.15.3:0/3314469113 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4060014480 con 0x7f406815d6c0 2023-03-03T06:25:19.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.152+0000 7f405ffff700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4048064ee0 0x7f4048067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:19.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.152+0000 7f405ffff700 1 -- 172.21.15.3:0/3314469113 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4060082b30 con 0x7f406815d6c0 2023-03-03T06:25:19.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.152+0000 7f4066d9d700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4048064ee0 0x7f4048067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:19.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.152+0000 7f4066d9d700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4048064ee0 0x7f4048067390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4058000c00 tx=0x7f4058008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:19.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.156+0000 7f405ffff700 1 -- 172.21.15.3:0/3314469113 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4060050890 con 0x7f406815d6c0 2023-03-03T06:25:19.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.372+0000 7f406c936700 1 -- 172.21.15.3:0/3314469113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4068073e90 con 0x7f406815d6c0 2023-03-03T06:25:19.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.372+0000 7f405ffff700 1 -- 172.21.15.3:0/3314469113 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4635 (secure 0 0 0) 0x7f4060053eb0 con 0x7f406815d6c0 2023-03-03T06:25:19.377 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:19.378 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2744699972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2744699972},{"type":"v1","addr":"172.21.15.3:6834","nonce":2744699972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/708054272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":708054272},{"type":"v1","addr":"172.21.15.3:6835","nonce":708054272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2935923413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2935923413},{"type":"v1","addr":"172.21.15.32:6837","nonce":2935923413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-03T06:25:13.798672+0000","modified":"2023-03-03T06:25:18.863903+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/691220172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":691220172},{"type":"v1","addr":"172.21.15.32:6836","nonce":691220172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-03-03T06:25:19.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.376+0000 7f406c936700 1 -- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4048064ee0 msgr2=0x7f4048067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:19.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.376+0000 7f406c936700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4048064ee0 0x7f4048067390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4058000c00 tx=0x7f4058008030 comp rx=0 tx=0).stop 2023-03-03T06:25:19.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.376+0000 7f406c936700 1 -- 172.21.15.3:0/3314469113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406815d6c0 msgr2=0x7f4068146ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:19.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.376+0000 7f406c936700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406815d6c0 0x7f4068146ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f406000efc0 tx=0x7f406000c0c0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.376+0000 7f406c936700 1 -- 172.21.15.3:0/3314469113 shutdown_connections 2023-03-03T06:25:19.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.376+0000 7f406c936700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4048064ee0 0x7f4048067390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.376+0000 7f406c936700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f406815d6c0 0x7f4068146ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.376+0000 7f406c936700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4068159120 0x7f40680c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.376+0000 7f406c936700 1 --2- 172.21.15.3:0/3314469113 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f40680ce890 0x7f40680c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.376+0000 7f406c936700 1 -- 172.21.15.3:0/3314469113 >> 172.21.15.3:0/3314469113 conn(0x7f4068069d70 msgr2=0x7f40680cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:19.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.376+0000 7f406c936700 1 -- 172.21.15.3:0/3314469113 shutdown_connections 2023-03-03T06:25:19.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.380+0000 7f406c936700 1 -- 172.21.15.3:0/3314469113 wait complete. 2023-03-03T06:25:19.385 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2023-03-03T06:25:19.416 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:25:19.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.624+0000 7fbbdec43700 1 Processor -- start 2023-03-03T06:25:19.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.624+0000 7fbbdec43700 1 -- start start 2023-03-03T06:25:19.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.624+0000 7fbbdec43700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbbd815a260 0x7fbbd815e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:19.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.624+0000 7fbbdec43700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd80cc090 con 0x7fbbd806e5c0 2023-03-03T06:25:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.624+0000 7fbbdec43700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd815dd40 con 0x7fbbd806def0 2023-03-03T06:25:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.624+0000 7fbbdec43700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd815e300 con 0x7fbbd815a260 2023-03-03T06:25:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbde442700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbbd806e5c0 0x7fbbd806e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45384/0 (socket says 172.21.15.3:45384) 2023-03-03T06:25:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbde442700 1 -- 172.21.15.3:0/103474244 learned_addr learned my addr 172.21.15.3:0/103474244 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbdd440700 1 --1- 172.21.15.3:0/103474244 >> v1:172.21.15.32:6789/0 conn(0x7fbbd806def0 0x7fbbd80cb980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:57768/0 (socket says 172.21.15.3:57768) 2023-03-03T06:25:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbddc41700 1 --2- 172.21.15.3:0/103474244 >> v2:172.21.15.32:3300/0 conn(0x7fbbd815a260 0x7fbbd815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:19.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbdcc3f700 1 -- 172.21.15.3:0/103474244 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446135153 0 0) 0x7fbbd80cc090 con 0x7fbbd806e5c0 2023-03-03T06:25:19.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbdcc3f700 1 -- 172.21.15.3:0/103474244 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc40035d0 con 0x7fbbd806e5c0 2023-03-03T06:25:19.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbdcc3f700 1 -- 172.21.15.3:0/103474244 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539089750 0 0) 0x7fbbd815dd40 con 0x7fbbd806def0 2023-03-03T06:25:19.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbddc41700 1 -- 172.21.15.3:0/103474244 >> v1:172.21.15.32:6789/0 conn(0x7fbbd806def0 legacy=0x7fbbd80cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:19.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbddc41700 1 -- 172.21.15.3:0/103474244 >> v1:172.21.15.3:6789/0 conn(0x7fbbd806e5c0 legacy=0x7fbbd806e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:19.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbddc41700 1 -- 172.21.15.3:0/103474244 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd815ecd0 con 0x7fbbd815a260 2023-03-03T06:25:19.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbddc41700 1 --2- 172.21.15.3:0/103474244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd815a260 0x7fbbd815e790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009d90 tx=0x7fbbd400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c9b6ef7ceb75aaf server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbdcc3f700 1 -- 172.21.15.3:0/103474244 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400e070 con 0x7fbbd815a260 2023-03-03T06:25:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbdcc3f700 1 -- 172.21.15.3:0/103474244 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd400bb20 con 0x7fbbd815a260 2023-03-03T06:25:19.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.628+0000 7fbbdcc3f700 1 -- 172.21.15.3:0/103474244 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400bed0 con 0x7fbbd815a260 2023-03-03T06:25:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.632+0000 7fbbdec43700 1 -- 172.21.15.3:0/103474244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd815a260 msgr2=0x7fbbd815e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.632+0000 7fbbdec43700 1 --2- 172.21.15.3:0/103474244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd815a260 0x7fbbd815e790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009d90 tx=0x7fbbd400b330 comp rx=0 tx=0).stop 2023-03-03T06:25:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.632+0000 7fbbdec43700 1 -- 172.21.15.3:0/103474244 shutdown_connections 2023-03-03T06:25:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.632+0000 7fbbdec43700 1 --2- 172.21.15.3:0/103474244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd815a260 0x7fbbd815e790 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.632+0000 7fbbdec43700 1 -- 172.21.15.3:0/103474244 >> 172.21.15.3:0/103474244 conn(0x7fbbd8069d70 msgr2=0x7fbbd806a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.632+0000 7fbbdec43700 1 -- 172.21.15.3:0/103474244 shutdown_connections 2023-03-03T06:25:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.632+0000 7fbbdec43700 1 -- 172.21.15.3:0/103474244 wait complete. 2023-03-03T06:25:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdec43700 1 Processor -- start 2023-03-03T06:25:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdec43700 1 -- start start 2023-03-03T06:25:19.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdec43700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbd806def0 0x7fbbd81f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:19.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdec43700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd806e5c0 0x7fbbd80b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:19.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdec43700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd815a260 0x7fbbd80b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:19.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdec43700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80b30c0 con 0x7fbbd806e5c0 2023-03-03T06:25:19.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdec43700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80b33a0 con 0x7fbbd815a260 2023-03-03T06:25:19.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdec43700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80b3680 con 0x7fbbd806def0 2023-03-03T06:25:19.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdd440700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd806e5c0 0x7fbbd80b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:19.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdd440700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd806e5c0 0x7fbbd80b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36122/0 (socket says 172.21.15.3:36122) 2023-03-03T06:25:19.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdd440700 1 -- 172.21.15.3:0/3695609262 learned_addr learned my addr 172.21.15.3:0/3695609262 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:19.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbde442700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd815a260 0x7fbbd80b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:19.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbddc41700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbd806def0 0x7fbbd81f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:19.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdd440700 1 -- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbd806def0 msgr2=0x7fbbd81f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:19.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdd440700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbd806def0 0x7fbbd81f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdd440700 1 -- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd815a260 msgr2=0x7fbbd80b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:19.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.636+0000 7fbbdd440700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd815a260 0x7fbbd80b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.640+0000 7fbbdd440700 1 -- 172.21.15.3:0/3695609262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd4009a70 con 0x7fbbd806e5c0 2023-03-03T06:25:19.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.640+0000 7fbbddc41700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbd806def0 0x7fbbd81f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:19.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.640+0000 7fbbdd440700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd806e5c0 0x7fbbd80b2670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800c4b0 tx=0x7fbbc800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.640+0000 7fbbce7fc700 1 -- 172.21.15.3:0/3695609262 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8004ca0 con 0x7fbbd806e5c0 2023-03-03T06:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.640+0000 7fbbce7fc700 1 -- 172.21.15.3:0/3695609262 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc8004e00 con 0x7fbbd806e5c0 2023-03-03T06:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.640+0000 7fbbce7fc700 1 -- 172.21.15.3:0/3695609262 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc80072d0 con 0x7fbbd806e5c0 2023-03-03T06:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.640+0000 7fbbdec43700 1 -- 172.21.15.3:0/3695609262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd80c9fc0 con 0x7fbbd806e5c0 2023-03-03T06:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.640+0000 7fbbdec43700 1 -- 172.21.15.3:0/3695609262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd815b640 con 0x7fbbd806e5c0 2023-03-03T06:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.644+0000 7fbbdec43700 1 -- 172.21.15.3:0/3695609262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd80ca2a0 con 0x7fbbd806e5c0 2023-03-03T06:25:19.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.644+0000 7fbbce7fc700 1 -- 172.21.15.3:0/3695609262 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fbbc801a020 con 0x7fbbd806e5c0 2023-03-03T06:25:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.648+0000 7fbbce7fc700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbbbc064ee0 0x7fbbbc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.648+0000 7fbbce7fc700 1 -- 172.21.15.3:0/3695609262 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbbc8081db0 con 0x7fbbd806e5c0 2023-03-03T06:25:19.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.648+0000 7fbbddc41700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbbbc064ee0 0x7fbbbc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:19.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.648+0000 7fbbddc41700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbbbc064ee0 0x7fbbbc067390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4014040 tx=0x7fbbd401d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:19.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.652+0000 7fbbce7fc700 1 -- 172.21.15.3:0/3695609262 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fbbc8086070 con 0x7fbbd806e5c0 2023-03-03T06:25:19.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.827+0000 7fbbdec43700 1 -- 172.21.15.3:0/3695609262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbd8073e90 con 0x7fbbd806e5c0 2023-03-03T06:25:19.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.831+0000 7fbbce7fc700 1 -- 172.21.15.3:0/3695609262 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19556 (secure 0 0 0) 0x7fbbc804fbc0 con 0x7fbbd806e5c0 2023-03-03T06:25:19.836 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:19.836 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:18.863350+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:25:19.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.835+0000 7fbbdec43700 1 -- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbbbc064ee0 msgr2=0x7fbbbc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:19.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.835+0000 7fbbdec43700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbbbc064ee0 0x7fbbbc067390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4014040 tx=0x7fbbd401d040 comp rx=0 tx=0).stop 2023-03-03T06:25:19.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.835+0000 7fbbdec43700 1 -- 172.21.15.3:0/3695609262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd806e5c0 msgr2=0x7fbbd80b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:19.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.835+0000 7fbbdec43700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd806e5c0 0x7fbbd80b2670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800c4b0 tx=0x7fbbc800c7c0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.835+0000 7fbbdec43700 1 -- 172.21.15.3:0/3695609262 shutdown_connections 2023-03-03T06:25:19.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.835+0000 7fbbdec43700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbbbc064ee0 0x7fbbbc067390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.835+0000 7fbbdec43700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbd815a260 0x7fbbd80b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.835+0000 7fbbdec43700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd806e5c0 0x7fbbd80b2670 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.835+0000 7fbbdec43700 1 --2- 172.21.15.3:0/3695609262 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbbd806def0 0x7fbbd81f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:19.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.835+0000 7fbbdec43700 1 -- 172.21.15.3:0/3695609262 >> 172.21.15.3:0/3695609262 conn(0x7fbbd8069d70 msgr2=0x7fbbd815cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:19.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.835+0000 7fbbdec43700 1 -- 172.21.15.3:0/3695609262 shutdown_connections 2023-03-03T06:25:19.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:19.835+0000 7fbbdec43700 1 -- 172.21.15.3:0/3695609262 wait complete. 2023-03-03T06:25:19.872 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:25:20.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.075+0000 7f6019e6f700 1 Processor -- start 2023-03-03T06:25:20.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.075+0000 7f6019e6f700 1 -- start start 2023-03-03T06:25:20.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.079+0000 7f6019e6f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60140b0cd0 0x7f60140c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:20.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.079+0000 7f6019e6f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60140b1120 con 0x7f60140cb1b0 2023-03-03T06:25:20.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f6019e6f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601415e190 con 0x7f60140ca280 2023-03-03T06:25:20.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f6019e6f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601415a7d0 con 0x7f60140b0cd0 2023-03-03T06:25:20.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f6018e6d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f60140cb1b0 0x7f60140cb580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45388/0 (socket says 172.21.15.3:45388) 2023-03-03T06:25:20.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f6018e6d700 1 -- 172.21.15.3:0/1110370346 learned_addr learned my addr 172.21.15.3:0/1110370346 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:20.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f600bfff700 1 --2- 172.21.15.3:0/1110370346 >> v2:172.21.15.32:3300/0 conn(0x7f60140b0cd0 0x7f60140c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:20.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f600b7fe700 1 -- 172.21.15.3:0/1110370346 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877494419 0 0) 0x7f60140b1120 con 0x7f60140cb1b0 2023-03-03T06:25:20.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f600b7fe700 1 -- 172.21.15.3:0/1110370346 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ffc0035d0 con 0x7f60140cb1b0 2023-03-03T06:25:20.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f600bfff700 1 -- 172.21.15.3:0/1110370346 >> v1:172.21.15.32:6789/0 conn(0x7f60140ca280 legacy=0x7f60140ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f600bfff700 1 -- 172.21.15.3:0/1110370346 >> v1:172.21.15.3:6789/0 conn(0x7f60140cb1b0 legacy=0x7f60140cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f600bfff700 1 -- 172.21.15.3:0/1110370346 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601415e470 con 0x7f60140b0cd0 2023-03-03T06:25:20.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f600bfff700 1 --2- 172.21.15.3:0/1110370346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60140b0cd0 0x7f60140c9d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6004009850 tx=0x7f6004009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff0222cd4e710f9d server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:20.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f600b7fe700 1 -- 172.21.15.3:0/1110370346 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004011830 con 0x7f60140b0cd0 2023-03-03T06:25:20.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f600b7fe700 1 -- 172.21.15.3:0/1110370346 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6004011990 con 0x7f60140b0cd0 2023-03-03T06:25:20.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.083+0000 7f600b7fe700 1 -- 172.21.15.3:0/1110370346 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004011d40 con 0x7f60140b0cd0 2023-03-03T06:25:20.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.087+0000 7f6019e6f700 1 -- 172.21.15.3:0/1110370346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60140b0cd0 msgr2=0x7f60140c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.087+0000 7f6019e6f700 1 --2- 172.21.15.3:0/1110370346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60140b0cd0 0x7f60140c9d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6004009850 tx=0x7f6004009b60 comp rx=0 tx=0).stop 2023-03-03T06:25:20.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.087+0000 7f6019e6f700 1 -- 172.21.15.3:0/1110370346 shutdown_connections 2023-03-03T06:25:20.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.087+0000 7f6019e6f700 1 --2- 172.21.15.3:0/1110370346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60140b0cd0 0x7f60140c9d40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.087+0000 7f6019e6f700 1 -- 172.21.15.3:0/1110370346 >> 172.21.15.3:0/1110370346 conn(0x7f6014070a30 msgr2=0x7f6014070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:20.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.087+0000 7f6019e6f700 1 -- 172.21.15.3:0/1110370346 shutdown_connections 2023-03-03T06:25:20.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.087+0000 7f6019e6f700 1 -- 172.21.15.3:0/1110370346 wait complete. 2023-03-03T06:25:20.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.087+0000 7f6019e6f700 1 Processor -- start 2023-03-03T06:25:20.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.087+0000 7f6019e6f700 1 -- start start 2023-03-03T06:25:20.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.087+0000 7f6019e6f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60140b0cd0 0x7f60140c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:20.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f6019e6f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60140ca280 0x7f60140c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:20.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f6019e6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60140cb1b0 0x7f6014146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:20.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f6019e6f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60140c2640 con 0x7f60140ca280 2023-03-03T06:25:20.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f6019e6f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60141579e0 con 0x7f60140cb1b0 2023-03-03T06:25:20.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f6019e6f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60141e68e0 con 0x7f60140b0cd0 2023-03-03T06:25:20.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f600bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60140ca280 0x7f60140c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:20.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f600bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60140ca280 0x7f60140c3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36126/0 (socket says 172.21.15.3:36126) 2023-03-03T06:25:20.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f600bfff700 1 -- 172.21.15.3:0/1671439265 learned_addr learned my addr 172.21.15.3:0/1671439265 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:20.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f6018e6d700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60140b0cd0 0x7f60140c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:20.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f601966e700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60140cb1b0 0x7f6014146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:20.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f600bfff700 1 -- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60140b0cd0 msgr2=0x7f60140c2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f600bfff700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60140b0cd0 0x7f60140c2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f600bfff700 1 -- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60140cb1b0 msgr2=0x7f6014146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f600bfff700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60140cb1b0 0x7f6014146700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f600bfff700 1 -- 172.21.15.3:0/1671439265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60040094b0 con 0x7f60140ca280 2023-03-03T06:25:20.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f600bfff700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60140ca280 0x7f60140c3dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6004000c00 tx=0x7f6004004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f60097fa700 1 -- 172.21.15.3:0/1671439265 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004019ab0 con 0x7f60140ca280 2023-03-03T06:25:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f60097fa700 1 -- 172.21.15.3:0/1671439265 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600401e070 con 0x7f60140ca280 2023-03-03T06:25:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f60097fa700 1 -- 172.21.15.3:0/1671439265 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004019e90 con 0x7f60140ca280 2023-03-03T06:25:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f6019e6f700 1 -- 172.21.15.3:0/1671439265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601415ea30 con 0x7f60140ca280 2023-03-03T06:25:20.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.091+0000 7f6019e6f700 1 -- 172.21.15.3:0/1671439265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601415e750 con 0x7f60140ca280 2023-03-03T06:25:20.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.095+0000 7f6019e6f700 1 -- 172.21.15.3:0/1671439265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60140af7e0 con 0x7f60140ca280 2023-03-03T06:25:20.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.095+0000 7f60097fa700 1 -- 172.21.15.3:0/1671439265 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f600400f050 con 0x7f60140ca280 2023-03-03T06:25:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.099+0000 7f60097fa700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6000064ee0 0x7f6000067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:20.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.099+0000 7f60097fa700 1 -- 172.21.15.3:0/1671439265 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f60040832e0 con 0x7f60140ca280 2023-03-03T06:25:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.099+0000 7f6018e6d700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6000064ee0 0x7f6000067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:20.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.099+0000 7f6018e6d700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6000064ee0 0x7f6000067390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6010001560 tx=0x7f6010009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:20.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.103+0000 7f60097fa700 1 -- 172.21.15.3:0/1671439265 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f60040510a0 con 0x7f60140ca280 2023-03-03T06:25:20.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.283+0000 7f6019e6f700 1 -- 172.21.15.3:0/1671439265 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6014073e90 con 0x7f60140ca280 2023-03-03T06:25:20.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.287+0000 7f60097fa700 1 -- 172.21.15.3:0/1671439265 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19556 (secure 0 0 0) 0x7f6004006ce0 con 0x7f60140ca280 2023-03-03T06:25:20.292 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:20.292 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:18.863350+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:25:20.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.291+0000 7f6019e6f700 1 -- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6000064ee0 msgr2=0x7f6000067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.291+0000 7f6019e6f700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6000064ee0 0x7f6000067390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6010001560 tx=0x7f6010009040 comp rx=0 tx=0).stop 2023-03-03T06:25:20.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.291+0000 7f6019e6f700 1 -- 172.21.15.3:0/1671439265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60140ca280 msgr2=0x7f60140c3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.291+0000 7f6019e6f700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60140ca280 0x7f60140c3dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6004000c00 tx=0x7f6004004770 comp rx=0 tx=0).stop 2023-03-03T06:25:20.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.295+0000 7f6019e6f700 1 -- 172.21.15.3:0/1671439265 shutdown_connections 2023-03-03T06:25:20.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.295+0000 7f6019e6f700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6000064ee0 0x7f6000067390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.295+0000 7f6019e6f700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60140cb1b0 0x7f6014146700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.295+0000 7f6019e6f700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60140ca280 0x7f60140c3dd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.295+0000 7f6019e6f700 1 --2- 172.21.15.3:0/1671439265 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60140b0cd0 0x7f60140c2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.295+0000 7f6019e6f700 1 -- 172.21.15.3:0/1671439265 >> 172.21.15.3:0/1671439265 conn(0x7f6014070a30 msgr2=0x7f60140ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:20.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.295+0000 7f6019e6f700 1 -- 172.21.15.3:0/1671439265 shutdown_connections 2023-03-03T06:25:20.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.295+0000 7f6019e6f700 1 -- 172.21.15.3:0/1671439265 wait complete. 2023-03-03T06:25:20.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-03T06:25:20.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.551+0000 7f5c2235d700 1 Processor -- start 2023-03-03T06:25:20.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.551+0000 7f5c2235d700 1 -- start start 2023-03-03T06:25:20.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c2235d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c1c15c750 0x7f5c1c158440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:20.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c2235d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1c0cdd30 con 0x7f5c1c0aee70 2023-03-03T06:25:20.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c2235d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1c158980 con 0x7f5c1c06a780 2023-03-03T06:25:20.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c2235d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0ca4d0 con 0x7f5c1c15c750 2023-03-03T06:25:20.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c20b5a700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5c1c0aee70 0x7f5c1c15c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35892/0 (socket says 172.21.15.3:35892) 2023-03-03T06:25:20.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c20b5a700 1 -- 172.21.15.3:0/1943247207 learned_addr learned my addr 172.21.15.3:0/1943247207 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:20.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c21b5c700 1 --2- 172.21.15.3:0/1943247207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c1c15c750 0x7f5c1c158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:20.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c13fff700 1 -- 172.21.15.3:0/1943247207 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223136356 0 0) 0x7f5c1c0cdd30 con 0x7f5c1c0aee70 2023-03-03T06:25:20.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c13fff700 1 -- 172.21.15.3:0/1943247207 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c080035d0 con 0x7f5c1c0aee70 2023-03-03T06:25:20.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c21b5c700 1 -- 172.21.15.3:0/1943247207 >> v1:172.21.15.32:6789/0 conn(0x7f5c1c06a780 legacy=0x7f5c1c0c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c21b5c700 1 -- 172.21.15.3:0/1943247207 >> v1:172.21.15.3:6789/0 conn(0x7f5c1c0aee70 legacy=0x7f5c1c15c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c21b5c700 1 -- 172.21.15.3:0/1943247207 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c0cd6e0 con 0x7f5c1c15c750 2023-03-03T06:25:20.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c21b5c700 1 --2- 172.21.15.3:0/1943247207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c1c15c750 0x7f5c1c158440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009dc0 tx=0x7f5c1400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7256e6c786c78f server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:20.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c13fff700 1 -- 172.21.15.3:0/1943247207 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400e070 con 0x7f5c1c15c750 2023-03-03T06:25:20.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c13fff700 1 -- 172.21.15.3:0/1943247207 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1400bb20 con 0x7f5c1c15c750 2023-03-03T06:25:20.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.555+0000 7f5c13fff700 1 -- 172.21.15.3:0/1943247207 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c14012430 con 0x7f5c1c15c750 2023-03-03T06:25:20.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.559+0000 7f5c2235d700 1 -- 172.21.15.3:0/1943247207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c1c15c750 msgr2=0x7f5c1c158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.559+0000 7f5c2235d700 1 --2- 172.21.15.3:0/1943247207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c1c15c750 0x7f5c1c158440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009dc0 tx=0x7f5c1400b330 comp rx=0 tx=0).stop 2023-03-03T06:25:20.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.559+0000 7f5c2235d700 1 -- 172.21.15.3:0/1943247207 shutdown_connections 2023-03-03T06:25:20.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.559+0000 7f5c2235d700 1 --2- 172.21.15.3:0/1943247207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c1c15c750 0x7f5c1c158440 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.559+0000 7f5c2235d700 1 -- 172.21.15.3:0/1943247207 >> 172.21.15.3:0/1943247207 conn(0x7f5c1c070a30 msgr2=0x7f5c1c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:20.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.559+0000 7f5c2235d700 1 -- 172.21.15.3:0/1943247207 shutdown_connections 2023-03-03T06:25:20.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.559+0000 7f5c2235d700 1 -- 172.21.15.3:0/1943247207 wait complete. 2023-03-03T06:25:20.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.563+0000 7f5c2235d700 1 Processor -- start 2023-03-03T06:25:20.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.563+0000 7f5c2235d700 1 -- start start 2023-03-03T06:25:20.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.563+0000 7f5c2235d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c1c06a780 0x7f5c1c1485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:20.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.563+0000 7f5c2235d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c1c0aee70 0x7f5c1c0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.563+0000 7f5c2235d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c1c15c750 0x7f5c1c0c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.563+0000 7f5c2235d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c14be50 con 0x7f5c1c06a780 2023-03-03T06:25:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.563+0000 7f5c2235d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0c4fa0 con 0x7f5c1c15c750 2023-03-03T06:25:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.563+0000 7f5c2235d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0c5280 con 0x7f5c1c0aee70 2023-03-03T06:25:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c2135b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c1c06a780 0x7f5c1c1485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c2135b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c1c06a780 0x7f5c1c1485f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44356/0 (socket says 172.21.15.3:44356) 2023-03-03T06:25:20.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c2135b700 1 -- 172.21.15.3:0/845636567 learned_addr learned my addr 172.21.15.3:0/845636567 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:20.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c20b5a700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c1c0aee70 0x7f5c1c0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:20.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c21b5c700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c1c15c750 0x7f5c1c0c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:20.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c2135b700 1 -- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c1c0aee70 msgr2=0x7f5c1c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c2135b700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c1c0aee70 0x7f5c1c0ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c2135b700 1 -- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c1c15c750 msgr2=0x7f5c1c0c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c2135b700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c1c15c750 0x7f5c1c0c2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c2135b700 1 -- 172.21.15.3:0/845636567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c14009a70 con 0x7f5c1c06a780 2023-03-03T06:25:20.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c2135b700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c1c06a780 0x7f5c1c1485f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800c500 tx=0x7f5c1800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c11ffb700 1 -- 172.21.15.3:0/845636567 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c180052d0 con 0x7f5c1c06a780 2023-03-03T06:25:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c11ffb700 1 -- 172.21.15.3:0/845636567 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c18005430 con 0x7f5c1c06a780 2023-03-03T06:25:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c11ffb700 1 -- 172.21.15.3:0/845636567 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c180055a0 con 0x7f5c1c06a780 2023-03-03T06:25:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c2235d700 1 -- 172.21.15.3:0/845636567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c0ca7b0 con 0x7f5c1c06a780 2023-03-03T06:25:20.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.567+0000 7f5c2235d700 1 -- 172.21.15.3:0/845636567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c0cd9c0 con 0x7f5c1c06a780 2023-03-03T06:25:20.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.571+0000 7f5c2235d700 1 -- 172.21.15.3:0/845636567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c15cd90 con 0x7f5c1c06a780 2023-03-03T06:25:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.571+0000 7f5c11ffb700 1 -- 172.21.15.3:0/845636567 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5c18019030 con 0x7f5c1c06a780 2023-03-03T06:25:20.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.575+0000 7f5c11ffb700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5c00064ee0 0x7f5c00067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:20.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.575+0000 7f5c11ffb700 1 -- 172.21.15.3:0/845636567 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5c180815b0 con 0x7f5c1c06a780 2023-03-03T06:25:20.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.575+0000 7f5c20b5a700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5c00064ee0 0x7f5c00067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:20.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.575+0000 7f5c20b5a700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5c00064ee0 0x7f5c00067390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c071b60 tx=0x7f5c0c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:20.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.579+0000 7f5c11ffb700 1 -- 172.21.15.3:0/845636567 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5c1804f370 con 0x7f5c1c06a780 2023-03-03T06:25:20.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.791+0000 7f5c2235d700 1 -- 172.21.15.3:0/845636567 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c1c073e90 con 0x7f5c1c06a780 2023-03-03T06:25:20.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.791+0000 7f5c11ffb700 1 -- 172.21.15.3:0/845636567 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4635 (secure 0 0 0) 0x7f5c18052990 con 0x7f5c1c06a780 2023-03-03T06:25:20.797 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:20.798 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2744699972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2744699972},{"type":"v1","addr":"172.21.15.3:6834","nonce":2744699972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/708054272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":708054272},{"type":"v1","addr":"172.21.15.3:6835","nonce":708054272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2935923413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2935923413},{"type":"v1","addr":"172.21.15.32:6837","nonce":2935923413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-03T06:25:13.798672+0000","modified":"2023-03-03T06:25:18.863903+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/691220172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":691220172},{"type":"v1","addr":"172.21.15.32:6836","nonce":691220172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-03-03T06:25:20.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.795+0000 7f5c2235d700 1 -- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5c00064ee0 msgr2=0x7f5c00067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.795+0000 7f5c2235d700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5c00064ee0 0x7f5c00067390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c071b60 tx=0x7f5c0c009040 comp rx=0 tx=0).stop 2023-03-03T06:25:20.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.795+0000 7f5c2235d700 1 -- 172.21.15.3:0/845636567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c1c06a780 msgr2=0x7f5c1c1485f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:20.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.795+0000 7f5c2235d700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c1c06a780 0x7f5c1c1485f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800c500 tx=0x7f5c1800c8c0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.795+0000 7f5c2235d700 1 -- 172.21.15.3:0/845636567 shutdown_connections 2023-03-03T06:25:20.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.795+0000 7f5c2235d700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5c00064ee0 0x7f5c00067390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.795+0000 7f5c2235d700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c1c15c750 0x7f5c1c0c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.795+0000 7f5c2235d700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c1c0aee70 0x7f5c1c0ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.795+0000 7f5c2235d700 1 --2- 172.21.15.3:0/845636567 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c1c06a780 0x7f5c1c1485f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:20.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.795+0000 7f5c2235d700 1 -- 172.21.15.3:0/845636567 >> 172.21.15.3:0/845636567 conn(0x7f5c1c070a30 msgr2=0x7f5c1c06c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:20.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.799+0000 7f5c2235d700 1 -- 172.21.15.3:0/845636567 shutdown_connections 2023-03-03T06:25:20.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:20.799+0000 7f5c2235d700 1 -- 172.21.15.3:0/845636567 wait complete. 2023-03-03T06:25:20.804 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2023-03-03T06:25:21.235 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-03-03T06:25:13.798672+0000', 'modified': '2023-03-03T06:25:18.863903+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54179}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54179': {'gid': 54179, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6836/691220172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 691220172}, {'type': 'v1', 'addr': '172.21.15.32:6836', 'nonce': 691220172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-03-03T06:25:21.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-03-03T06:25:21.236 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-03-03T06:25:21.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.451+0000 7f6b00e8c700 1 Processor -- start 2023-03-03T06:25:21.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.451+0000 7f6b00e8c700 1 -- start start 2023-03-03T06:25:21.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.451+0000 7f6b00e8c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6afc0ce890 0x7f6afc0aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.451+0000 7f6b00e8c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6afc15d6c0 0x7f6afc0cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.451+0000 7f6b00e8c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6afc0cc590 con 0x7f6afc159120 2023-03-03T06:25:21.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.451+0000 7f6b00e8c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6afc15db20 con 0x7f6afc15d6c0 2023-03-03T06:25:21.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.451+0000 7f6b00e8c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc1594f0 con 0x7f6afc0ce890 2023-03-03T06:25:21.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.455+0000 7f6afb7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6afc159120 0x7f6afc0ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35898/0 (socket says 172.21.15.3:35898) 2023-03-03T06:25:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.455+0000 7f6afb7fe700 1 -- 172.21.15.3:0/3196810008 learned_addr learned my addr 172.21.15.3:0/3196810008 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.455+0000 7f6afbfff700 1 --2- 172.21.15.3:0/3196810008 >> v2:172.21.15.32:3300/0 conn(0x7f6afc15d6c0 0x7f6afc0cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.455+0000 7f6afaffd700 1 --2- 172.21.15.3:0/3196810008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6afc0ce890 0x7f6afc0aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.455+0000 7f6afa7fc700 1 -- 172.21.15.3:0/3196810008 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2212864054 0 0) 0x7f6aec001110 con 0x7f6afc159120 2023-03-03T06:25:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.455+0000 7f6afa7fc700 1 -- 172.21.15.3:0/3196810008 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae40035d0 con 0x7f6afc159120 2023-03-03T06:25:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.455+0000 7f6afbfff700 1 -- 172.21.15.3:0/3196810008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6afc0ce890 msgr2=0x7f6afc0aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.455+0000 7f6afbfff700 1 --2- 172.21.15.3:0/3196810008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6afc0ce890 0x7f6afc0aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.455+0000 7f6afbfff700 1 -- 172.21.15.3:0/3196810008 >> v1:172.21.15.3:6789/0 conn(0x7f6afc159120 legacy=0x7f6afc0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.455+0000 7f6afbfff700 1 -- 172.21.15.3:0/3196810008 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc0cc7d0 con 0x7f6afc15d6c0 2023-03-03T06:25:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.455+0000 7f6afaffd700 1 --2- 172.21.15.3:0/3196810008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6afc0ce890 0x7f6afc0aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:21.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.455+0000 7f6afbfff700 1 --2- 172.21.15.3:0/3196810008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6afc15d6c0 0x7f6afc0cc050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6af4009d90 tx=0x7f6af400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83eee8e3e43e0f22 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:21.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.459+0000 7f6afa7fc700 1 -- 172.21.15.3:0/3196810008 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af400e070 con 0x7f6afc15d6c0 2023-03-03T06:25:21.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.459+0000 7f6afa7fc700 1 -- 172.21.15.3:0/3196810008 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af400bb20 con 0x7f6afc15d6c0 2023-03-03T06:25:21.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.459+0000 7f6afa7fc700 1 -- 172.21.15.3:0/3196810008 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4012430 con 0x7f6afc15d6c0 2023-03-03T06:25:21.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.459+0000 7f6b00e8c700 1 -- 172.21.15.3:0/3196810008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6afc15d6c0 msgr2=0x7f6afc0cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.459+0000 7f6b00e8c700 1 --2- 172.21.15.3:0/3196810008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6afc15d6c0 0x7f6afc0cc050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6af4009d90 tx=0x7f6af400b330 comp rx=0 tx=0).stop 2023-03-03T06:25:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.459+0000 7f6b00e8c700 1 -- 172.21.15.3:0/3196810008 shutdown_connections 2023-03-03T06:25:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.459+0000 7f6b00e8c700 1 --2- 172.21.15.3:0/3196810008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6afc15d6c0 0x7f6afc0cc050 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.459+0000 7f6b00e8c700 1 --2- 172.21.15.3:0/3196810008 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6afc0ce890 0x7f6afc0aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.459+0000 7f6b00e8c700 1 -- 172.21.15.3:0/3196810008 >> 172.21.15.3:0/3196810008 conn(0x7f6afc069d70 msgr2=0x7f6afc06a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:21.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.459+0000 7f6b00e8c700 1 -- 172.21.15.3:0/3196810008 shutdown_connections 2023-03-03T06:25:21.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.459+0000 7f6b00e8c700 1 -- 172.21.15.3:0/3196810008 wait complete. 2023-03-03T06:25:21.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.463+0000 7f6b00e8c700 1 Processor -- start 2023-03-03T06:25:21.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.463+0000 7f6b00e8c700 1 -- start start 2023-03-03T06:25:21.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6b00e8c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6afc0ce890 0x7f6afc0b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6b00e8c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6afc159120 0x7f6afc0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6b00e8c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6afc15d6c0 0x7f6afc15ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6b00e8c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc159c80 con 0x7f6afc159120 2023-03-03T06:25:21.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6b00e8c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc159f60 con 0x7f6afc15d6c0 2023-03-03T06:25:21.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6b00e8c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc15a240 con 0x7f6afc0ce890 2023-03-03T06:25:21.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6afaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6afc159120 0x7f6afc0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6afaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6afc159120 0x7f6afc0c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44372/0 (socket says 172.21.15.3:44372) 2023-03-03T06:25:21.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6afaffd700 1 -- 172.21.15.3:0/51288108 learned_addr learned my addr 172.21.15.3:0/51288108 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:21.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6afb7fe700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6afc0ce890 0x7f6afc0b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6afbfff700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6afc15d6c0 0x7f6afc15ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6afaffd700 1 -- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6afc0ce890 msgr2=0x7f6afc0b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6afaffd700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6afc0ce890 0x7f6afc0b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6afaffd700 1 -- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6afc15d6c0 msgr2=0x7f6afc15ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6afaffd700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6afc15d6c0 0x7f6afc15ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6afaffd700 1 -- 172.21.15.3:0/51288108 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af00094b0 con 0x7f6afc159120 2023-03-03T06:25:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6afbfff700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6afc15d6c0 0x7f6afc15ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6afaffd700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6afc159120 0x7f6afc0c8d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6af000e9b0 tx=0x7f6af000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:21.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6ae3fff700 1 -- 172.21.15.3:0/51288108 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af000c850 con 0x7f6afc159120 2023-03-03T06:25:21.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6b00e8c700 1 -- 172.21.15.3:0/51288108 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af4009a70 con 0x7f6afc159120 2023-03-03T06:25:21.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6ae3fff700 1 -- 172.21.15.3:0/51288108 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af000c9b0 con 0x7f6afc159120 2023-03-03T06:25:21.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6ae3fff700 1 -- 172.21.15.3:0/51288108 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af000cd60 con 0x7f6afc159120 2023-03-03T06:25:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.467+0000 7f6b00e8c700 1 -- 172.21.15.3:0/51288108 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6afc15cbe0 con 0x7f6afc159120 2023-03-03T06:25:21.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.475+0000 7f6ae3fff700 1 -- 172.21.15.3:0/51288108 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6af0019030 con 0x7f6afc159120 2023-03-03T06:25:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.479+0000 7f6ae3fff700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6ae8064ee0 0x7f6ae8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.479+0000 7f6ae3fff700 1 -- 172.21.15.3:0/51288108 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6af0081f70 con 0x7f6afc159120 2023-03-03T06:25:21.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.479+0000 7f6afb7fe700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6ae8064ee0 0x7f6ae8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.479+0000 7f6b05378740 1 Processor -- start 2023-03-03T06:25:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.479+0000 7f6b05378740 1 -- start start 2023-03-03T06:25:21.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.479+0000 7f6afb7fe700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6ae8064ee0 0x7f6ae8067390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6aec007fd0 tx=0x7f6aec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.479+0000 7f6b05378740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258cae0 0x283af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.479+0000 7f6b05378740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258c710 0x2825a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.479+0000 7f6b05378740 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x25d7790 0x27f8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6b05378740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x26fbf00 con 0x258c710 2023-03-03T06:25:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6b05378740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x276bff0 con 0x258cae0 2023-03-03T06:25:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6b05378740 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x27652f0 con 0x25d7790 2023-03-03T06:25:21.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6afaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258c710 0x2825a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6afaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258c710 0x2825a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44378/0 (socket says 172.21.15.3:44378) 2023-03-03T06:25:21.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6afaffd700 1 -- 172.21.15.3:0/2046416764 learned_addr learned my addr 172.21.15.3:0/2046416764 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:21.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6afb7fe700 1 --2- 172.21.15.3:0/2046416764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258cae0 0x283af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6afbfff700 1 --2- 172.21.15.3:0/2046416764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x25d7790 0x27f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6afaffd700 1 -- 172.21.15.3:0/2046416764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x25d7790 msgr2=0x27f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:21.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6afaffd700 1 --2- 172.21.15.3:0/2046416764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x25d7790 0x27f8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6afaffd700 1 -- 172.21.15.3:0/2046416764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258cae0 msgr2=0x283af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:21.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6afaffd700 1 --2- 172.21.15.3:0/2046416764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258cae0 0x283af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6afaffd700 1 -- 172.21.15.3:0/2046416764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x278ac90 con 0x258c710 2023-03-03T06:25:21.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6afbfff700 1 --2- 172.21.15.3:0/2046416764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x25d7790 0x27f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:21.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6afaffd700 1 --2- 172.21.15.3:0/2046416764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258c710 0x2825a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6af0053610 tx=0x7f6af00533a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d603917bb3ec7e server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:21.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6ae17fa700 1 -- 172.21.15.3:0/2046416764 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0053000 con 0x258c710 2023-03-03T06:25:21.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6ae17fa700 1 -- 172.21.15.3:0/2046416764 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af005b570 con 0x258c710 2023-03-03T06:25:21.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6ae17fa700 1 -- 172.21.15.3:0/2046416764 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af004f700 con 0x258c710 2023-03-03T06:25:21.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6b05378740 1 -- 172.21.15.3:0/2046416764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258c710 msgr2=0x2825a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.483+0000 7f6b05378740 1 --2- 172.21.15.3:0/2046416764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258c710 0x2825a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6af0053610 tx=0x7f6af00533a0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 -- 172.21.15.3:0/2046416764 shutdown_connections 2023-03-03T06:25:21.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 --2- 172.21.15.3:0/2046416764 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x25d7790 0x27f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 --2- 172.21.15.3:0/2046416764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258c710 0x2825a40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 --2- 172.21.15.3:0/2046416764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258cae0 0x283af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 -- 172.21.15.3:0/2046416764 >> 172.21.15.3:0/2046416764 conn(0x258ceb0 msgr2=0x2825330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:21.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 -- 172.21.15.3:0/2046416764 shutdown_connections 2023-03-03T06:25:21.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 -- 172.21.15.3:0/2046416764 wait complete. 2023-03-03T06:25:21.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 -- start start 2023-03-03T06:25:21.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 Processor -- start 2023-03-03T06:25:21.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258c710 0x2852600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258cae0 0x2855410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x25d7790 0x2820510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x2611b20 con 0x25d7790 2023-03-03T06:25:21.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x283a7f0 con 0x258cae0 2023-03-03T06:25:21.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x2855950 con 0x258c710 2023-03-03T06:25:21.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6b05378740 10 client.0 _ll_register_callbacks cb 0x27c0d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-03T06:25:21.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x25d7790 0x2820510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x25d7790 0x2820510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44392/0 (socket says 172.21.15.3:44392) 2023-03-03T06:25:21.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6afbfff700 1 -- 172.21.15.3:0/3857582065 learned_addr learned my addr 172.21.15.3:0/3857582065 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:21.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6afb7fe700 1 --2- 172.21.15.3:0/3857582065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258c710 0x2852600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.487+0000 7f6afaffd700 1 --2- 172.21.15.3:0/3857582065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258cae0 0x2855410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6afbfff700 1 -- 172.21.15.3:0/3857582065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258c710 msgr2=0x2852600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6afbfff700 1 --2- 172.21.15.3:0/3857582065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258c710 0x2852600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6afbfff700 1 -- 172.21.15.3:0/3857582065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258cae0 msgr2=0x2855410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6afbfff700 1 --2- 172.21.15.3:0/3857582065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258cae0 0x2855410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6afbfff700 1 -- 172.21.15.3:0/3857582065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6afc15cbe0 con 0x25d7790 2023-03-03T06:25:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6afbfff700 1 --2- 172.21.15.3:0/3857582065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x25d7790 0x2820510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6afc06b210 tx=0x7f6af4004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576ecc64789fe2d5 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6ada7fc700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-03-03T06:25:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6ada7fc700 1 -- 172.21.15.3:0/3857582065 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4020070 con 0x25d7790 2023-03-03T06:25:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6ada7fc700 1 -- 172.21.15.3:0/3857582065 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6af40128f0 con 0x25d7790 2023-03-03T06:25:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6b05378740 1 -- 172.21.15.3:0/3857582065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x275e370 con 0x25d7790 2023-03-03T06:25:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6ada7fc700 1 -- 172.21.15.3:0/3857582065 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6af4012ab0 con 0x25d7790 2023-03-03T06:25:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6b05378740 10 client.4416 fetch_fsmap learned FSMap version 6 2023-03-03T06:25:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6b05378740 1 -- 172.21.15.3:0/3857582065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x25d89e0 con 0x25d7790 2023-03-03T06:25:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6ada7fc700 1 -- 172.21.15.3:0/3857582065 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x275e370 con 0x25d7790 2023-03-03T06:25:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6ada7fc700 1 -- 172.21.15.3:0/3857582065 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 6) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6af4012c10 con 0x25d7790 2023-03-03T06:25:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6b05378740 10 client.4416 fetch_fsmap finished waiting for FSMap version 6 2023-03-03T06:25:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6b05378740 10 client.4416 resolve_mds: resolved name 'b' to daemon mds.b 2023-03-03T06:25:21.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6b05378740 20 client.4416 populate_metadata read hostname 'smithi003' 2023-03-03T06:25:21.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6b05378740 1 --2- 172.21.15.3:0/3857582065 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x2820db0 0x28244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6b05378740 4 client.4416 mds_command: new command op to 54179 tid=0[{"prefix": "get_command_descriptions"}] 2023-03-03T06:25:21.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6b05378740 1 -- 172.21.15.3:0/3857582065 --> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x275e090 con 0x2820db0 2023-03-03T06:25:21.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.491+0000 7f6afb7fe700 1 --2- 172.21.15.3:0/3857582065 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x2820db0 0x28244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.495+0000 7f6afb7fe700 1 --2- 172.21.15.3:0/3857582065 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x2820db0 0x28244a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ae823a3715951ca server_cookie=74b5a44caf96a538 in_seq=0 out_seq=0 2023-03-03T06:25:21.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.495+0000 7f6ada7fc700 10 client.4416 ms_handle_connect on v2:172.21.15.32:6834/691220172 2023-03-03T06:25:21.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.495+0000 7f6ada7fc700 1 -- 172.21.15.3:0/3857582065 <== mds.0 v2:172.21.15.32:6834/691220172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18278 (crc 0 0 0) 0x275e090 con 0x2820db0 2023-03-03T06:25:21.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.495+0000 7f6ada7fc700 10 client.4416 handle_command_reply: tid=0 2023-03-03T06:25:21.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.495+0000 7f6b05378740 1 client.4416 shutdown 2023-03-03T06:25:21.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.499+0000 7f6b05378740 1 -- 172.21.15.3:0/3857582065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x25d7790 msgr2=0x2820510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.499+0000 7f6b05378740 1 --2- 172.21.15.3:0/3857582065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x25d7790 0x2820510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6afc06b210 tx=0x7f6af4004b60 comp rx=0 tx=0).stop 2023-03-03T06:25:21.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.499+0000 7f6b05378740 1 -- 172.21.15.3:0/3857582065 shutdown_connections 2023-03-03T06:25:21.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.499+0000 7f6b05378740 1 --2- 172.21.15.3:0/3857582065 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x2820db0 0x28244a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.499+0000 7f6b05378740 1 --2- 172.21.15.3:0/3857582065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x25d7790 0x2820510 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.499+0000 7f6b05378740 1 --2- 172.21.15.3:0/3857582065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258cae0 0x2855410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.499+0000 7f6b05378740 1 --2- 172.21.15.3:0/3857582065 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258c710 0x2852600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.499+0000 7f6ada7fc700 0 client.4416 ms_handle_reset on v2:172.21.15.32:6834/691220172 2023-03-03T06:25:21.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.499+0000 7f6b05378740 1 -- 172.21.15.3:0/3857582065 >> 172.21.15.3:0/3857582065 conn(0x258ceb0 msgr2=0x281cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:21.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.499+0000 7f6b05378740 1 -- 172.21.15.3:0/3857582065 shutdown_connections 2023-03-03T06:25:21.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.499+0000 7f6b05378740 1 -- 172.21.15.3:0/3857582065 wait complete. 2023-03-03T06:25:21.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.499+0000 7f6b05378740 20 client.4416 trim_cache size 0 max 16384 2023-03-03T06:25:21.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.519+0000 7f6b05378740 1 Processor -- start 2023-03-03T06:25:21.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.519+0000 7f6b05378740 1 -- start start 2023-03-03T06:25:21.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6b05378740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258ceb0 0x2819d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6b05378740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258c710 0x281a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6b05378740 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258cae0 0x285bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6b05378740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x2822e10 con 0x258ceb0 2023-03-03T06:25:21.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6b05378740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x26f5e30 con 0x258c710 2023-03-03T06:25:21.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6b05378740 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x2765ba0 con 0x258cae0 2023-03-03T06:25:21.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6afb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258ceb0 0x2819d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6afbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258cae0 0x285bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6afaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258c710 0x281a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6afb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258ceb0 0x2819d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44402/0 (socket says 172.21.15.3:44402) 2023-03-03T06:25:21.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6afb7fe700 1 -- 172.21.15.3:0/205482662 learned_addr learned my addr 172.21.15.3:0/205482662 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:21.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6afb7fe700 1 -- 172.21.15.3:0/205482662 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258cae0 msgr2=0x285bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:21.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6afb7fe700 1 --2- 172.21.15.3:0/205482662 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258cae0 0x285bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6afb7fe700 1 -- 172.21.15.3:0/205482662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258c710 msgr2=0x281a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:21.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6afb7fe700 1 --2- 172.21.15.3:0/205482662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258c710 0x281a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6afb7fe700 1 -- 172.21.15.3:0/205482662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc1599f0 con 0x258ceb0 2023-03-03T06:25:21.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6afaffd700 1 --2- 172.21.15.3:0/205482662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258c710 0x281a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:21.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6afb7fe700 1 --2- 172.21.15.3:0/205482662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258ceb0 0x2819d30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6aec007d60 tx=0x7f6aec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9843c4c9b14ca81f server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:21.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6ae1ffb700 1 -- 172.21.15.3:0/205482662 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec005d40 con 0x258ceb0 2023-03-03T06:25:21.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6ae1ffb700 1 -- 172.21.15.3:0/205482662 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec026ad0 con 0x258ceb0 2023-03-03T06:25:21.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.523+0000 7f6ae1ffb700 1 -- 172.21.15.3:0/205482662 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec005ea0 con 0x258ceb0 2023-03-03T06:25:21.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.527+0000 7f6b05378740 1 -- 172.21.15.3:0/205482662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258ceb0 msgr2=0x2819d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.527+0000 7f6b05378740 1 --2- 172.21.15.3:0/205482662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258ceb0 0x2819d30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6aec007d60 tx=0x7f6aec012000 comp rx=0 tx=0).stop 2023-03-03T06:25:21.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.527+0000 7f6b05378740 1 -- 172.21.15.3:0/205482662 shutdown_connections 2023-03-03T06:25:21.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.527+0000 7f6b05378740 1 --2- 172.21.15.3:0/205482662 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258cae0 0x285bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.527+0000 7f6b05378740 1 --2- 172.21.15.3:0/205482662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258c710 0x281a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.527+0000 7f6b05378740 1 --2- 172.21.15.3:0/205482662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258ceb0 0x2819d30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.527+0000 7f6b05378740 1 -- 172.21.15.3:0/205482662 >> 172.21.15.3:0/205482662 conn(0x2855580 msgr2=0x281aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.527+0000 7f6b05378740 1 -- 172.21.15.3:0/205482662 shutdown_connections 2023-03-03T06:25:21.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.527+0000 7f6b05378740 1 -- 172.21.15.3:0/205482662 wait complete. 2023-03-03T06:25:21.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6b05378740 1 -- start start 2023-03-03T06:25:21.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6b05378740 1 Processor -- start 2023-03-03T06:25:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6b05378740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258c710 0x28ab800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6b05378740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258cae0 0x28a2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6b05378740 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258ceb0 0x28a2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6b05378740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x28ae1c0 con 0x258cae0 2023-03-03T06:25:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6b05378740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x28a2eb0 con 0x258c710 2023-03-03T06:25:21.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6b05378740 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x28ae3a0 con 0x258ceb0 2023-03-03T06:25:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6b05378740 10 client.0 _ll_register_callbacks cb 0x272fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-03T06:25:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6afaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258cae0 0x28a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6afaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258cae0 0x28a2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44406/0 (socket says 172.21.15.3:44406) 2023-03-03T06:25:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6afaffd700 1 -- 172.21.15.3:0/2728158478 learned_addr learned my addr 172.21.15.3:0/2728158478 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6afbfff700 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258ceb0 0x28a2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6afb7fe700 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258c710 0x28ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6afaffd700 1 -- 172.21.15.3:0/2728158478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258ceb0 msgr2=0x28a2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:21.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.531+0000 7f6afaffd700 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258ceb0 0x28a2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6afaffd700 1 -- 172.21.15.3:0/2728158478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258c710 msgr2=0x28ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:21.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6afaffd700 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258c710 0x28ab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6afaffd700 1 -- 172.21.15.3:0/2728158478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x278ac90 con 0x258cae0 2023-03-03T06:25:21.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6afbfff700 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258ceb0 0x28a2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:25:21.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6afaffd700 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258cae0 0x28a2430 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6af0000c00 tx=0x7f6af005bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906dd411f3482b3 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:21.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6ada7fc700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-03-03T06:25:21.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6ada7fc700 1 -- 172.21.15.3:0/2728158478 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0052870 con 0x258cae0 2023-03-03T06:25:21.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6b05378740 1 -- 172.21.15.3:0/2728158478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6af005b570 con 0x258cae0 2023-03-03T06:25:21.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6ada7fc700 1 -- 172.21.15.3:0/2728158478 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6af00529d0 con 0x258cae0 2023-03-03T06:25:21.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6ada7fc700 1 -- 172.21.15.3:0/2728158478 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6af0052b90 con 0x258cae0 2023-03-03T06:25:21.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6b05378740 10 client.4422 fetch_fsmap learned FSMap version 6 2023-03-03T06:25:21.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6b05378740 1 -- 172.21.15.3:0/2728158478 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x7f6aec00f040 con 0x258cae0 2023-03-03T06:25:21.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6ada7fc700 1 -- 172.21.15.3:0/2728158478 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af005b570 con 0x258cae0 2023-03-03T06:25:21.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6ada7fc700 1 -- 172.21.15.3:0/2728158478 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 6) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6af0005a40 con 0x258cae0 2023-03-03T06:25:21.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6b05378740 10 client.4422 fetch_fsmap finished waiting for FSMap version 6 2023-03-03T06:25:21.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6b05378740 10 client.4422 resolve_mds: resolved name 'b' to daemon mds.b 2023-03-03T06:25:21.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6b05378740 20 client.4422 populate_metadata read hostname 'smithi003' 2023-03-03T06:25:21.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6b05378740 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x25d7790 0x28ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6b05378740 4 client.4422 mds_command: new command op to 54179 tid=0[{"prefix": "status"}] 2023-03-03T06:25:21.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6b05378740 1 -- 172.21.15.3:0/2728158478 --> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] -- command(tid 0: {"prefix": "status"}) v1 -- 0x2609260 con 0x25d7790 2023-03-03T06:25:21.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.535+0000 7f6afb7fe700 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x25d7790 0x28ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6afb7fe700 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x25d7790 0x28ac800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b9fb64896742538 server_cookie=d561e5e3c6f38b46 in_seq=0 out_seq=0 2023-03-03T06:25:21.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6ada7fc700 10 client.4422 ms_handle_connect on v2:172.21.15.32:6834/691220172 2023-03-03T06:25:21.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6ada7fc700 1 -- 172.21.15.3:0/2728158478 <== mds.0 v2:172.21.15.32:6834/691220172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x2609260 con 0x25d7790 2023-03-03T06:25:21.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6ada7fc700 10 client.4422 handle_command_reply: tid=0 2023-03-03T06:25:21.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b05378740 1 client.4422 shutdown 2023-03-03T06:25:21.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b05378740 1 -- 172.21.15.3:0/2728158478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258cae0 msgr2=0x28a2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b05378740 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258cae0 0x28a2430 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6af0000c00 tx=0x7f6af005bb80 comp rx=0 tx=0).stop 2023-03-03T06:25:21.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b05378740 1 -- 172.21.15.3:0/2728158478 shutdown_connections 2023-03-03T06:25:21.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b05378740 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x25d7790 0x28ac800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b05378740 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x258ceb0 0x28a2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b05378740 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x258cae0 0x28a2430 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b05378740 1 --2- 172.21.15.3:0/2728158478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x258c710 0x28ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b05378740 1 -- 172.21.15.3:0/2728158478 >> 172.21.15.3:0/2728158478 conn(0x2855580 msgr2=0x2819d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:21.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6ada7fc700 0 client.4422 ms_handle_reset on v2:172.21.15.32:6834/691220172 2023-03-03T06:25:21.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b05378740 1 -- 172.21.15.3:0/2728158478 shutdown_connections 2023-03-03T06:25:21.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b05378740 1 -- 172.21.15.3:0/2728158478 wait complete. 2023-03-03T06:25:21.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b05378740 20 client.4422 trim_cache size 0 max 16384 2023-03-03T06:25:21.546 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-03-03T06:25:21.546 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "7d496254-7268-405f-a1d8-c8033dc8659a", 2023-03-03T06:25:21.547 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-03-03T06:25:21.547 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 54179, 2023-03-03T06:25:21.547 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-03-03T06:25:21.547 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-03-03T06:25:21.547 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-03-03T06:25:21.547 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 7.7326106699999997, 2023-03-03T06:25:21.547 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 6, 2023-03-03T06:25:21.548 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 28, 2023-03-03T06:25:21.548 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 27, 2023-03-03T06:25:21.548 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 14.130656677999999 2023-03-03T06:25:21.548 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-03-03T06:25:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b00e8c700 1 -- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6ae8064ee0 msgr2=0x7f6ae8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b00e8c700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6ae8064ee0 0x7f6ae8067390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6aec007fd0 tx=0x7f6aec009040 comp rx=0 tx=0).stop 2023-03-03T06:25:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b00e8c700 1 -- 172.21.15.3:0/51288108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6afc159120 msgr2=0x7f6afc0c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.539+0000 7f6b00e8c700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6afc159120 0x7f6afc0c8d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6af000e9b0 tx=0x7f6af000ed70 comp rx=0 tx=0).stop 2023-03-03T06:25:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.543+0000 7f6b00e8c700 1 -- 172.21.15.3:0/51288108 shutdown_connections 2023-03-03T06:25:21.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.543+0000 7f6b00e8c700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6ae8064ee0 0x7f6ae8067390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.543+0000 7f6b00e8c700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6afc15d6c0 0x7f6afc15ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.543+0000 7f6b00e8c700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6afc159120 0x7f6afc0c8d50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.543+0000 7f6b00e8c700 1 --2- 172.21.15.3:0/51288108 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6afc0ce890 0x7f6afc0b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.543+0000 7f6b00e8c700 1 -- 172.21.15.3:0/51288108 >> 172.21.15.3:0/51288108 conn(0x7f6afc069d70 msgr2=0x7f6afc06dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.543+0000 7f6b00e8c700 1 -- 172.21.15.3:0/51288108 shutdown_connections 2023-03-03T06:25:21.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.543+0000 7f6b00e8c700 1 -- 172.21.15.3:0/51288108 wait complete. 2023-03-03T06:25:21.580 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-03-03T06:25:21.581 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-03-03T06:25:21.582 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-03-03T06:25:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.787+0000 7f5ecb59e700 1 Processor -- start 2023-03-03T06:25:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.787+0000 7f5ecb59e700 1 -- start start 2023-03-03T06:25:21.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5ecb59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ecc1608d0 0x7f5ecc0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5ecb59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5ecc160d20 0x7f5ecc0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5ecb59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ecc160440 con 0x7f5ecc153670 2023-03-03T06:25:21.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5ecb59e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ecc0cd3e0 con 0x7f5ecc160d20 2023-03-03T06:25:21.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5ecb59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0cd6c0 con 0x7f5ecc1608d0 2023-03-03T06:25:21.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5eca59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5ecc153670 0x7f5ecc071350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35910/0 (socket says 172.21.15.3:35910) 2023-03-03T06:25:21.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5eca59c700 1 -- 172.21.15.3:0/1007486201 learned_addr learned my addr 172.21.15.3:0/1007486201 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:21.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5ecad9d700 1 --2- 172.21.15.3:0/1007486201 >> v2:172.21.15.3:3300/0 conn(0x7f5ecc160d20 0x7f5ecc0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5ec9d9b700 1 --2- 172.21.15.3:0/1007486201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ecc1608d0 0x7f5ecc0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5ec959a700 1 -- 172.21.15.3:0/1007486201 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556105639 0 0) 0x7f5ecc160440 con 0x7f5ecc153670 2023-03-03T06:25:21.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5ec959a700 1 -- 172.21.15.3:0/1007486201 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5eb40035d0 con 0x7f5ecc153670 2023-03-03T06:25:21.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5ecad9d700 1 -- 172.21.15.3:0/1007486201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ecc1608d0 msgr2=0x7f5ecc0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:21.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5ecad9d700 1 --2- 172.21.15.3:0/1007486201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ecc1608d0 0x7f5ecc0cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5ecad9d700 1 -- 172.21.15.3:0/1007486201 >> v1:172.21.15.3:6789/0 conn(0x7f5ecc153670 legacy=0x7f5ecc071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.791+0000 7f5ecad9d700 1 -- 172.21.15.3:0/1007486201 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc1600d0 con 0x7f5ecc160d20 2023-03-03T06:25:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.795+0000 7f5ecad9d700 1 --2- 172.21.15.3:0/1007486201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc160d20 0x7f5ecc0cfb60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009850 tx=0x7f5ec4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1c5e304268dcbe server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.795+0000 7f5ec959a700 1 -- 172.21.15.3:0/1007486201 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4011830 con 0x7f5ecc160d20 2023-03-03T06:25:21.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.795+0000 7f5ec959a700 1 -- 172.21.15.3:0/1007486201 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec4011990 con 0x7f5ecc160d20 2023-03-03T06:25:21.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.795+0000 7f5ec959a700 1 -- 172.21.15.3:0/1007486201 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4011d00 con 0x7f5ecc160d20 2023-03-03T06:25:21.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.795+0000 7f5ecb59e700 1 -- 172.21.15.3:0/1007486201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc160d20 msgr2=0x7f5ecc0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.795+0000 7f5ecb59e700 1 --2- 172.21.15.3:0/1007486201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc160d20 0x7f5ecc0cfb60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009850 tx=0x7f5ec4009b60 comp rx=0 tx=0).stop 2023-03-03T06:25:21.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.799+0000 7f5ecb59e700 1 -- 172.21.15.3:0/1007486201 shutdown_connections 2023-03-03T06:25:21.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.799+0000 7f5ecb59e700 1 --2- 172.21.15.3:0/1007486201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc160d20 0x7f5ecc0cfb60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.799+0000 7f5ecb59e700 1 --2- 172.21.15.3:0/1007486201 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ecc1608d0 0x7f5ecc0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.799+0000 7f5ecb59e700 1 -- 172.21.15.3:0/1007486201 >> 172.21.15.3:0/1007486201 conn(0x7f5ecc06e5c0 msgr2=0x7f5ecc06e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:21.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.799+0000 7f5ecb59e700 1 -- 172.21.15.3:0/1007486201 shutdown_connections 2023-03-03T06:25:21.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.799+0000 7f5ecb59e700 1 -- 172.21.15.3:0/1007486201 wait complete. 2023-03-03T06:25:21.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5ecb59e700 1 Processor -- start 2023-03-03T06:25:21.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5ecb59e700 1 -- start start 2023-03-03T06:25:21.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5ecb59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc153670 0x7f5ecc154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5ecb59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ecc1608d0 0x7f5ecc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5ecb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ecc160d20 0x7f5ecc14b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5ecb59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0b6a90 con 0x7f5ecc153670 2023-03-03T06:25:21.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5ecb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0b6d70 con 0x7f5ecc160d20 2023-03-03T06:25:21.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5ecb59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0b5440 con 0x7f5ecc1608d0 2023-03-03T06:25:21.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5eca59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc153670 0x7f5ecc154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5eca59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc153670 0x7f5ecc154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44430/0 (socket says 172.21.15.3:44430) 2023-03-03T06:25:21.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5eca59c700 1 -- 172.21.15.3:0/433784112 learned_addr learned my addr 172.21.15.3:0/433784112 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:21.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5ecad9d700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ecc160d20 0x7f5ecc14b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5ec9d9b700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ecc1608d0 0x7f5ecc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5eca59c700 1 -- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ecc1608d0 msgr2=0x7f5ecc0b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5eca59c700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ecc1608d0 0x7f5ecc0b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5eca59c700 1 -- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ecc160d20 msgr2=0x7f5ecc14b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:21.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5eca59c700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ecc160d20 0x7f5ecc14b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:21.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5eca59c700 1 -- 172.21.15.3:0/433784112 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec40094b0 con 0x7f5ecc153670 2023-03-03T06:25:21.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.803+0000 7f5ec9d9b700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ecc1608d0 0x7f5ecc0b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:21.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.807+0000 7f5eca59c700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc153670 0x7f5ecc154450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00cc20 tx=0x7f5ebc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:21.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.807+0000 7f5ebaffd700 1 -- 172.21.15.3:0/433784112 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc00a960 con 0x7f5ecc153670 2023-03-03T06:25:21.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.807+0000 7f5ebaffd700 1 -- 172.21.15.3:0/433784112 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc00aac0 con 0x7f5ecc153670 2023-03-03T06:25:21.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.807+0000 7f5ecb59e700 1 -- 172.21.15.3:0/433784112 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ecc0cdb30 con 0x7f5ecc153670 2023-03-03T06:25:21.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.807+0000 7f5ebaffd700 1 -- 172.21.15.3:0/433784112 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc00ae50 con 0x7f5ecc153670 2023-03-03T06:25:21.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.807+0000 7f5ecb59e700 1 -- 172.21.15.3:0/433784112 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ecc0cd9a0 con 0x7f5ecc153670 2023-03-03T06:25:21.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.807+0000 7f5ecb59e700 1 -- 172.21.15.3:0/433784112 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ecc0cf200 con 0x7f5ecc153670 2023-03-03T06:25:21.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.811+0000 7f5ebaffd700 1 -- 172.21.15.3:0/433784112 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5ebc019030 con 0x7f5ecc153670 2023-03-03T06:25:21.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.811+0000 7f5ebaffd700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5eac064f10 0x7f5eac0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:21.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.815+0000 7f5ebaffd700 1 -- 172.21.15.3:0/433784112 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5ebc081b20 con 0x7f5ecc153670 2023-03-03T06:25:21.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.815+0000 7f5ec9d9b700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5eac064f10 0x7f5eac0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:21.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.815+0000 7f5ec9d9b700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5eac064f10 0x7f5eac0673c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc14eca0 tx=0x7f5ec0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:21.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:21.819+0000 7f5ebaffd700 1 -- 172.21.15.3:0/433784112 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5ebc04f8e0 con 0x7f5ecc153670 2023-03-03T06:25:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.035+0000 7f5ecb59e700 1 -- 172.21.15.3:0/433784112 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5ecc073e90 con 0x7f5ecc153670 2023-03-03T06:25:22.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.035+0000 7f5ebaffd700 1 -- 172.21.15.3:0/433784112 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5ebc052f00 con 0x7f5ecc153670 2023-03-03T06:25:22.041 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-03-03T06:25:22.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.039+0000 7f5ecb59e700 1 -- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5eac064f10 msgr2=0x7f5eac0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:22.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.039+0000 7f5ecb59e700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5eac064f10 0x7f5eac0673c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc14eca0 tx=0x7f5ec0008040 comp rx=0 tx=0).stop 2023-03-03T06:25:22.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.039+0000 7f5ecb59e700 1 -- 172.21.15.3:0/433784112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc153670 msgr2=0x7f5ecc154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:22.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.039+0000 7f5ecb59e700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc153670 0x7f5ecc154450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00cc20 tx=0x7f5ebc00cc50 comp rx=0 tx=0).stop 2023-03-03T06:25:22.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.039+0000 7f5ecb59e700 1 -- 172.21.15.3:0/433784112 shutdown_connections 2023-03-03T06:25:22.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.039+0000 7f5ecb59e700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5eac064f10 0x7f5eac0673c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:22.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.039+0000 7f5ecb59e700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ecc160d20 0x7f5ecc14b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:22.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.039+0000 7f5ecb59e700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ecc1608d0 0x7f5ecc0b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:22.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.039+0000 7f5ecb59e700 1 --2- 172.21.15.3:0/433784112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc153670 0x7f5ecc154450 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:22.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.039+0000 7f5ecb59e700 1 -- 172.21.15.3:0/433784112 >> 172.21.15.3:0/433784112 conn(0x7f5ecc06e5c0 msgr2=0x7f5ecc0d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:22.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.039+0000 7f5ecb59e700 1 -- 172.21.15.3:0/433784112 shutdown_connections 2023-03-03T06:25:22.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:22.039+0000 7f5ecb59e700 1 -- 172.21.15.3:0/433784112 wait complete. 2023-03-03T06:25:22.079 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-03-03T06:25:22.079 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-03-03T06:25:22.080 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-03-03T06:25:22.080 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-03-03T06:25:22.080 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-03-03T06:25:22.080 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-03-03T06:25:22.080 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-03-03T06:25:22.081 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-03-03T06:25:22.081 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-03-03T06:25:22.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:22.082 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-03-03T06:25:22.102 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-03T06:25:22.102 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-03T06:25:22.103 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-03-03T06:25:22.103 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:22.103 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-03-03T06:25:22.103 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:22.103 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:25:22.104 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:22.104 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-03-03T06:25:22.104 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-03-03T06:25:22.104 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:22.104 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-03-03T06:25:22.105 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-03-03T06:25:22.105 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42370sec preferred_lft 42370sec 2023-03-03T06:25:22.105 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2023-03-03T06:25:22.105 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:22.105 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:25:22.105 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:22.106 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-03-03T06:25:22.106 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:25:22.106 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:22.106 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-03-03T06:25:22.106 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-03-03T06:25:22.106 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:69:76:c1:31 brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:22.106 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-03-03T06:25:22.107 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:22.108 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-03-03T06:25:22.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:22.108 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:22.109 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:22.109 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2023-03-03T06:25:22.109 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2023-03-03T06:25:22.109 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2023-03-03T06:25:22.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-03-03T06:25:22.110 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:22.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:22.305 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-03-03T06:25:22.347 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-03-03T06:25:22.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:22.352 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-03-03T06:25:22.437 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-03-03T06:25:22.438 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-03-03T06:25:22.438 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-03-03T06:25:22.438 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-03-03T06:25:22.438 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-03-03T06:25:22.438 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-03-03T06:25:22.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:22.440 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:22.440 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:22.440 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-03-03T06:25:22.441 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-03-03T06:25:22.441 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-03-03T06:25:22.441 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:22.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:22.613 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-03-03T06:25:22.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:22.636 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2023-03-03T06:25:22.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:22.704 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:22.704 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:22.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-03-03T06:25:22.705 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-03-03T06:25:22.705 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:22.851 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-03-03T06:25:22.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:22.852 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:22.852 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:22.852 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-03-03T06:25:22.852 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-03-03T06:25:22.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-03-03T06:25:22.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-03-03T06:25:22.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-03-03T06:25:22.853 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:23.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:23.074 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:23.074 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:23.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-03-03T06:25:23.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2023-03-03T06:25:23.075 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:23.171 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-03-03T06:25:23.171 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-03T06:25:23.172 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-03-03T06:25:23.190 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-03-03T06:25:23.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-03T06:25:23.191 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-03-03T06:25:23.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-03-03T06:25:23.348 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-03T06:25:23.349 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-03T06:25:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-03T06:25:23.434 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-03T06:25:23.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:23.436 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-03-03T06:25:23.498 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-03-03T06:25:23.499 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_mountpoint=/ --client_fs=cephfs) 2023-03-03T06:25:23.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-03-03T06:25:23.596 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-03T06:25:23.597 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-03T06:25:23.684 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-03T06:25:23.687 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-03T06:25:23.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:23.688 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-03-03T06:25:23.739 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:2023-03-03T06:25:23.735+0000 7ff017c7d640 -1 init, newargv = 0x560ccb837430 newargc=15 2023-03-03T06:25:23.739 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[37589]: starting ceph client 2023-03-03T06:25:23.760 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[37589]: starting fuse 2023-03-03T06:25:24.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2023-03-03T06:25:24.899 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-03T06:25:24.899 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-03T06:25:24.985 INFO:teuthology.orchestra.run.smithi003.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-03T06:25:24.989 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-03T06:25:24.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:24.991 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2023-03-03T06:25:25.054 INFO:teuthology.orchestra.run.smithi003.stdout:48 2023-03-03T06:25:25.055 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [48] 2023-03-03T06:25:25.055 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-03-03T06:25:25.056 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2023-03-03T06:25:25.056 DEBUG:teuthology.orchestra.run.smithi003:> import re 2023-03-03T06:25:25.056 DEBUG:teuthology.orchestra.run.smithi003:> import os 2023-03-03T06:25:25.057 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2023-03-03T06:25:25.057 DEBUG:teuthology.orchestra.run.smithi003:> 2023-03-03T06:25:25.057 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2023-03-03T06:25:25.058 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-03-03T06:25:25.058 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2023-03-03T06:25:25.058 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-03-03T06:25:25.059 DEBUG:teuthology.orchestra.run.smithi003:> 2023-03-03T06:25:25.059 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2023-03-03T06:25:25.059 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2023-03-03T06:25:25.060 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2023-03-03T06:25:25.060 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2023-03-03T06:25:25.060 DEBUG:teuthology.orchestra.run.smithi003:> 2023-03-03T06:25:25.061 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2023-03-03T06:25:25.061 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-03-03T06:25:25.062 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2023-03-03T06:25:25.063 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-03-03T06:25:25.063 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2023-03-03T06:25:25.063 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2023-03-03T06:25:25.064 DEBUG:teuthology.orchestra.run.smithi003:> return f 2023-03-03T06:25:25.064 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-03-03T06:25:25.064 DEBUG:teuthology.orchestra.run.smithi003:> 2023-03-03T06:25:25.065 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.admin")) 2023-03-03T06:25:25.065 DEBUG:teuthology.orchestra.run.smithi003:> ' 2023-03-03T06:25:25.239 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.admin.37589.asok 2023-03-03T06:25:25.425 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.37589.asok 2023-03-03T06:25:25.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:25.426 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.37589.asok status 2023-03-03T06:25:25.654 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-03-03T06:25:25.654 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2023-03-03T06:25:25.654 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "472aaef4ec3b3a2c29f73ca67854785a910f1f9b", 2023-03-03T06:25:25.654 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 18.0.0-2739-g472aaef4 (472aaef4ec3b3a2c29f73ca67854785a910f1f9b) reef (dev)", 2023-03-03T06:25:25.655 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "admin", 2023-03-03T06:25:25.655 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2023-03-03T06:25:25.655 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-03-03T06:25:25.655 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "37589", 2023-03-03T06:25:25.655 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2023-03-03T06:25:25.655 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:25:25.655 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2023-03-03T06:25:25.655 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2023-03-03T06:25:25.656 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 54302, 2023-03-03T06:25:25.656 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2023-03-03T06:25:25.656 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2023-03-03T06:25:25.656 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2023-03-03T06:25:25.656 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 54302 2023-03-03T06:25:25.656 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:25:25.656 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-03-03T06:25:25.657 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-03-03T06:25:25.657 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-03-03T06:25:25.657 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2213257731 2023-03-03T06:25:25.657 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:25:25.657 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:25:25.657 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2023-03-03T06:25:25.657 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2023-03-03T06:25:25.658 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2023-03-03T06:25:25.658 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2213257731 2023-03-03T06:25:25.658 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:25:25.658 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.54302 192.168.0.1:0/2213257731", 2023-03-03T06:25:25.658 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/2213257731", 2023-03-03T06:25:25.658 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2023-03-03T06:25:25.658 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 6, 2023-03-03T06:25:25.658 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 28, 2023-03-03T06:25:25.659 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2023-03-03T06:25:25.659 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2023-03-03T06:25:25.659 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2023-03-03T06:25:25.659 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-03-03T06:25:25.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:25.677 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2023-03-03T06:25:25.677 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-03-03T06:25:25.698 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2023-03-03T06:25:25.699 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-03-03T06:25:25.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:25.700 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-03-03T06:25:25.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:25.789 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-03-03T06:25:25.910 INFO:teuthology.orchestra.run.smithi003.stdout:# file: . 2023-03-03T06:25:25.910 INFO:teuthology.orchestra.run.smithi003.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-03-03T06:25:25.910 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:25.912 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:25:25.913 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:25:25.933 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:25:26.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.187+0000 7f7dc67e8700 1 Processor -- start 2023-03-03T06:25:26.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.191+0000 7f7dc67e8700 1 -- start start 2023-03-03T06:25:26.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.191+0000 7f7dc67e8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7dc0153670 0x7f7dc0071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:26.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7dc57e6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7dc0153670 0x7f7dc0071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:26.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7dc67e8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc0160440 con 0x7f7dc00cfd30 2023-03-03T06:25:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7dc5fe7700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7dc00cfd30 0x7f7dc00d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35914/0 (socket says 172.21.15.3:35914) 2023-03-03T06:25:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7dc5fe7700 1 -- 172.21.15.3:0/1315513629 learned_addr learned my addr 172.21.15.3:0/1315513629 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7dc67e8700 1 -- 172.21.15.3:0/1315513629 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc00cd3e0 con 0x7f7dc01608d0 2023-03-03T06:25:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7dc67e8700 1 -- 172.21.15.3:0/1315513629 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc00cd6c0 con 0x7f7dc0153670 2023-03-03T06:25:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7db7fff700 1 -- 172.21.15.3:0/1315513629 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242399509 0 0) 0x7f7dc0160440 con 0x7f7dc00cfd30 2023-03-03T06:25:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7db7fff700 1 -- 172.21.15.3:0/1315513629 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dac0035d0 con 0x7f7dc00cfd30 2023-03-03T06:25:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7db7fff700 1 -- 172.21.15.3:0/1315513629 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906725153 0 0) 0x7f7dc00cd3e0 con 0x7f7dc01608d0 2023-03-03T06:25:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7dc57e6700 1 -- 172.21.15.3:0/1315513629 >> v1:172.21.15.32:6789/0 conn(0x7f7dc01608d0 legacy=0x7f7dc00cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7dc57e6700 1 -- 172.21.15.3:0/1315513629 >> v1:172.21.15.3:6789/0 conn(0x7f7dc00cfd30 legacy=0x7f7dc00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7dc57e6700 1 -- 172.21.15.3:0/1315513629 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc01600d0 con 0x7f7dc0153670 2023-03-03T06:25:26.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7dc57e6700 1 --2- 172.21.15.3:0/1315513629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dc0153670 0x7f7dc0071350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009880 tx=0x7f7dbc009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94313ff009f6aeed server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:26.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7db7fff700 1 -- 172.21.15.3:0/1315513629 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db80019f0 con 0x7f7dc0153670 2023-03-03T06:25:26.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7db7fff700 1 -- 172.21.15.3:0/1315513629 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc010590 con 0x7f7dc0153670 2023-03-03T06:25:26.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.195+0000 7f7db7fff700 1 -- 172.21.15.3:0/1315513629 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc010900 con 0x7f7dc0153670 2023-03-03T06:25:26.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.199+0000 7f7dc67e8700 1 -- 172.21.15.3:0/1315513629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dc0153670 msgr2=0x7f7dc0071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:26.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.199+0000 7f7dc67e8700 1 --2- 172.21.15.3:0/1315513629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dc0153670 0x7f7dc0071350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009880 tx=0x7f7dbc009b90 comp rx=0 tx=0).stop 2023-03-03T06:25:26.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.199+0000 7f7dc67e8700 1 -- 172.21.15.3:0/1315513629 shutdown_connections 2023-03-03T06:25:26.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.199+0000 7f7dc67e8700 1 --2- 172.21.15.3:0/1315513629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dc0153670 0x7f7dc0071350 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.199+0000 7f7dc67e8700 1 -- 172.21.15.3:0/1315513629 >> 172.21.15.3:0/1315513629 conn(0x7f7dc006e5c0 msgr2=0x7f7dc006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:26.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.199+0000 7f7dc67e8700 1 -- 172.21.15.3:0/1315513629 shutdown_connections 2023-03-03T06:25:26.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.199+0000 7f7dc67e8700 1 -- 172.21.15.3:0/1315513629 wait complete. 2023-03-03T06:25:26.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.203+0000 7f7dc67e8700 1 Processor -- start 2023-03-03T06:25:26.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.203+0000 7f7dc67e8700 1 -- start start 2023-03-03T06:25:26.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.203+0000 7f7dc67e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc00cfd30 0x7f7dc00b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:26.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.203+0000 7f7dc67e8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dc0153670 0x7f7dc014c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:26.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.203+0000 7f7dc67e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dc01608d0 0x7f7dc00d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:26.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.203+0000 7f7dc67e8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00cca30 con 0x7f7dc0153670 2023-03-03T06:25:26.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.203+0000 7f7dc67e8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00cb990 con 0x7f7dc00cfd30 2023-03-03T06:25:26.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.203+0000 7f7dc67e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00cbc70 con 0x7f7dc01608d0 2023-03-03T06:25:26.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.203+0000 7f7dc4fe5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dc0153670 0x7f7dc014c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:26.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.203+0000 7f7dc4fe5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dc0153670 0x7f7dc014c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44452/0 (socket says 172.21.15.3:44452) 2023-03-03T06:25:26.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.203+0000 7f7dc4fe5700 1 -- 172.21.15.3:0/4206642324 learned_addr learned my addr 172.21.15.3:0/4206642324 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:26.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7dc5fe7700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dc01608d0 0x7f7dc00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:26.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7dc57e6700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc00cfd30 0x7f7dc00b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:26.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7dc4fe5700 1 -- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dc01608d0 msgr2=0x7f7dc00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:26.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7dc4fe5700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dc01608d0 0x7f7dc00d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7dc4fe5700 1 -- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc00cfd30 msgr2=0x7f7dc00b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:26.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7dc4fe5700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc00cfd30 0x7f7dc00b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7dc4fe5700 1 -- 172.21.15.3:0/4206642324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0094b0 con 0x7f7dc0153670 2023-03-03T06:25:26.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7dc4fe5700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dc0153670 0x7f7dc014c9d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7db000c450 tx=0x7f7db000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:26.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7db5ffb700 1 -- 172.21.15.3:0/4206642324 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db00054e0 con 0x7f7dc0153670 2023-03-03T06:25:26.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7db5ffb700 1 -- 172.21.15.3:0/4206642324 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db00036a0 con 0x7f7dc0153670 2023-03-03T06:25:26.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7db5ffb700 1 -- 172.21.15.3:0/4206642324 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0003a50 con 0x7f7dc0153670 2023-03-03T06:25:26.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7dc67e8700 1 -- 172.21.15.3:0/4206642324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc00cd9a0 con 0x7f7dc0153670 2023-03-03T06:25:26.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7dc67e8700 1 -- 172.21.15.3:0/4206642324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc0071890 con 0x7f7dc0153670 2023-03-03T06:25:26.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.207+0000 7f7dc67e8700 1 -- 172.21.15.3:0/4206642324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc00cdb30 con 0x7f7dc0153670 2023-03-03T06:25:26.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.211+0000 7f7db5ffb700 1 -- 172.21.15.3:0/4206642324 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7db0019030 con 0x7f7dc0153670 2023-03-03T06:25:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.215+0000 7f7db5ffb700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7da4064ee0 0x7f7da4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:26.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.215+0000 7f7db5ffb700 1 -- 172.21.15.3:0/4206642324 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7db00815f0 con 0x7f7dc0153670 2023-03-03T06:25:26.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.215+0000 7f7dc57e6700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7da4064ee0 0x7f7da4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:26.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.215+0000 7f7dc57e6700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7da4064ee0 0x7f7da4067390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009850 tx=0x7f7dbc010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.219+0000 7f7db5ffb700 1 -- 172.21.15.3:0/4206642324 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7db004f3b0 con 0x7f7dc0153670 2023-03-03T06:25:26.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.411+0000 7f7dc67e8700 1 -- 172.21.15.3:0/4206642324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dc0073e90 con 0x7f7dc0153670 2023-03-03T06:25:26.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.415+0000 7f7db5ffb700 1 -- 172.21.15.3:0/4206642324 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19556 (secure 0 0 0) 0x7f7db0001160 con 0x7f7dc0153670 2023-03-03T06:25:26.419 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:26.419 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:18.863350+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:25:26.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.419+0000 7f7dc67e8700 1 -- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7da4064ee0 msgr2=0x7f7da4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:26.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.419+0000 7f7dc67e8700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7da4064ee0 0x7f7da4067390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009850 tx=0x7f7dbc010000 comp rx=0 tx=0).stop 2023-03-03T06:25:26.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.419+0000 7f7dc67e8700 1 -- 172.21.15.3:0/4206642324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dc0153670 msgr2=0x7f7dc014c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:26.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.419+0000 7f7dc67e8700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dc0153670 0x7f7dc014c9d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7db000c450 tx=0x7f7db000c810 comp rx=0 tx=0).stop 2023-03-03T06:25:26.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.419+0000 7f7dc67e8700 1 -- 172.21.15.3:0/4206642324 shutdown_connections 2023-03-03T06:25:26.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.419+0000 7f7dc67e8700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7da4064ee0 0x7f7da4067390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.419+0000 7f7dc67e8700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dc01608d0 0x7f7dc00d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.419+0000 7f7dc67e8700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dc0153670 0x7f7dc014c9d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.419+0000 7f7dc67e8700 1 --2- 172.21.15.3:0/4206642324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dc00cfd30 0x7f7dc00b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.419+0000 7f7dc67e8700 1 -- 172.21.15.3:0/4206642324 >> 172.21.15.3:0/4206642324 conn(0x7f7dc006e5c0 msgr2=0x7f7dc00cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:26.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.419+0000 7f7dc67e8700 1 -- 172.21.15.3:0/4206642324 shutdown_connections 2023-03-03T06:25:26.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.419+0000 7f7dc67e8700 1 -- 172.21.15.3:0/4206642324 wait complete. 2023-03-03T06:25:26.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-03T06:25:11.468534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-03T06:25:12.813283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-03T06:25:16.382800+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-03T06:25:26.455 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:25:26.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.663+0000 7f41e568c700 1 Processor -- start 2023-03-03T06:25:26.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.663+0000 7f41e568c700 1 -- start start 2023-03-03T06:25:26.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41e568c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41e0156ba0 0x7f41e0071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:26.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41e568c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41e015d6e0 0x7f41e00cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:26.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41e568c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e00cf3b0 con 0x7f41e00cfd80 2023-03-03T06:25:26.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41e568c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e0156f70 con 0x7f41e0156ba0 2023-03-03T06:25:26.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41e568c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00cd320 con 0x7f41e015d6e0 2023-03-03T06:25:26.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41dffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41e0156ba0 0x7f41e0071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:26.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41dffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41e0156ba0 0x7f41e0071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44458/0 (socket says 172.21.15.3:44458) 2023-03-03T06:25:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41dffff700 1 -- 172.21.15.3:0/1710677217 learned_addr learned my addr 172.21.15.3:0/1710677217 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41e4e8b700 1 --1- 172.21.15.3:0/1710677217 >> v1:172.21.15.3:6789/0 conn(0x7f41e00cfd80 0x7f41e00d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35918/0 (socket says 172.21.15.3:35918) 2023-03-03T06:25:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41df7fe700 1 --2- 172.21.15.3:0/1710677217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41e015d6e0 0x7f41e00cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41dffff700 1 -- 172.21.15.3:0/1710677217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41e015d6e0 msgr2=0x7f41e00cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41dffff700 1 --2- 172.21.15.3:0/1710677217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41e015d6e0 0x7f41e00cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41dffff700 1 -- 172.21.15.3:0/1710677217 >> v1:172.21.15.3:6789/0 conn(0x7f41e00cfd80 legacy=0x7f41e00d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:26.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41dffff700 1 -- 172.21.15.3:0/1710677217 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e00cd600 con 0x7f41e0156ba0 2023-03-03T06:25:26.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41deffd700 1 -- 172.21.15.3:0/1710677217 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320572973 0 0) 0x7f41e00cf3b0 con 0x7f41e00cfd80 2023-03-03T06:25:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41dffff700 1 --2- 172.21.15.3:0/1710677217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41e0156ba0 0x7f41e0071280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009910 tx=0x7f41d0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fd4b4eccc3fbc1 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.667+0000 7f41deffd700 1 -- 172.21.15.3:0/1710677217 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000ba60 con 0x7f41e0156ba0 2023-03-03T06:25:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.671+0000 7f41deffd700 1 -- 172.21.15.3:0/1710677217 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d000bbc0 con 0x7f41e0156ba0 2023-03-03T06:25:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.671+0000 7f41deffd700 1 -- 172.21.15.3:0/1710677217 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0009e60 con 0x7f41e0156ba0 2023-03-03T06:25:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.671+0000 7f41e568c700 1 -- 172.21.15.3:0/1710677217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41e0156ba0 msgr2=0x7f41e0071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.671+0000 7f41e568c700 1 --2- 172.21.15.3:0/1710677217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41e0156ba0 0x7f41e0071280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009910 tx=0x7f41d0009c20 comp rx=0 tx=0).stop 2023-03-03T06:25:26.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.671+0000 7f41e568c700 1 -- 172.21.15.3:0/1710677217 shutdown_connections 2023-03-03T06:25:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.671+0000 7f41e568c700 1 --2- 172.21.15.3:0/1710677217 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41e015d6e0 0x7f41e00cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.671+0000 7f41e568c700 1 --2- 172.21.15.3:0/1710677217 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41e0156ba0 0x7f41e0071280 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.671+0000 7f41e568c700 1 -- 172.21.15.3:0/1710677217 >> 172.21.15.3:0/1710677217 conn(0x7f41e006e5c0 msgr2=0x7f41e006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.671+0000 7f41e568c700 1 -- 172.21.15.3:0/1710677217 shutdown_connections 2023-03-03T06:25:26.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.671+0000 7f41e568c700 1 -- 172.21.15.3:0/1710677217 wait complete. 2023-03-03T06:25:26.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41e568c700 1 Processor -- start 2023-03-03T06:25:26.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41e568c700 1 -- start start 2023-03-03T06:25:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41e568c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41e00cfd80 0x7f41e00b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:26.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41e568c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41e0156ba0 0x7f41e00c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41e568c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41e015d6e0 0x7f41e00cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41e568c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00b75b0 con 0x7f41e00cfd80 2023-03-03T06:25:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41e568c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00c9420 con 0x7f41e0156ba0 2023-03-03T06:25:26.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41e568c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00b7800 con 0x7f41e015d6e0 2023-03-03T06:25:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41dffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41e00cfd80 0x7f41e00b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41dffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41e00cfd80 0x7f41e00b85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44474/0 (socket says 172.21.15.3:44474) 2023-03-03T06:25:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41dffff700 1 -- 172.21.15.3:0/3320625198 learned_addr learned my addr 172.21.15.3:0/3320625198 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:26.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41df7fe700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41e0156ba0 0x7f41e00c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:26.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41e4e8b700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41e015d6e0 0x7f41e00cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:26.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41dffff700 1 -- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41e015d6e0 msgr2=0x7f41e00cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:26.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41dffff700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41e015d6e0 0x7f41e00cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41dffff700 1 -- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41e0156ba0 msgr2=0x7f41e00c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:26.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41dffff700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41e0156ba0 0x7f41e00c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41dffff700 1 -- 172.21.15.3:0/3320625198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d0009570 con 0x7f41e00cfd80 2023-03-03T06:25:26.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41e4e8b700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41e015d6e0 0x7f41e00cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:26.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.675+0000 7f41dffff700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41e00cfd80 0x7f41e00b85c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f41e006f190 tx=0x7f41d0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:26.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.679+0000 7f41dcff9700 1 -- 172.21.15.3:0/3320625198 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0004510 con 0x7f41e00cfd80 2023-03-03T06:25:26.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.679+0000 7f41dcff9700 1 -- 172.21.15.3:0/3320625198 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d001f070 con 0x7f41e00cfd80 2023-03-03T06:25:26.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.679+0000 7f41dcff9700 1 -- 172.21.15.3:0/3320625198 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d00038e0 con 0x7f41e00cfd80 2023-03-03T06:25:26.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.679+0000 7f41e568c700 1 -- 172.21.15.3:0/3320625198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e00717c0 con 0x7f41e00cfd80 2023-03-03T06:25:26.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.679+0000 7f41e568c700 1 -- 172.21.15.3:0/3320625198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e00cd8e0 con 0x7f41e00cfd80 2023-03-03T06:25:26.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.679+0000 7f41e568c700 1 -- 172.21.15.3:0/3320625198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e00cef40 con 0x7f41e00cfd80 2023-03-03T06:25:26.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.679+0000 7f41dcff9700 1 -- 172.21.15.3:0/3320625198 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f41d000f050 con 0x7f41e00cfd80 2023-03-03T06:25:26.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.683+0000 7f41dcff9700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f41cc064f10 0x7f41cc0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:26.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.683+0000 7f41dcff9700 1 -- 172.21.15.3:0/3320625198 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f41d0021070 con 0x7f41e00cfd80 2023-03-03T06:25:26.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.683+0000 7f41df7fe700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f41cc064f10 0x7f41cc0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:26.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.683+0000 7f41df7fe700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f41cc064f10 0x7f41cc0673c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41e0155b60 tx=0x7f41d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.687+0000 7f41dcff9700 1 -- 172.21.15.3:0/3320625198 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f41d0051570 con 0x7f41e00cfd80 2023-03-03T06:25:26.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.859+0000 7f41e568c700 1 -- 172.21.15.3:0/3320625198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41e0073e90 con 0x7f41e00cfd80 2023-03-03T06:25:26.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.863+0000 7f41dcff9700 1 -- 172.21.15.3:0/3320625198 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41d00150a0 con 0x7f41e00cfd80 2023-03-03T06:25:26.866 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:25:26.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.863+0000 7f41e568c700 1 -- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f41cc064f10 msgr2=0x7f41cc0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:26.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.863+0000 7f41e568c700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f41cc064f10 0x7f41cc0673c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41e0155b60 tx=0x7f41d4008040 comp rx=0 tx=0).stop 2023-03-03T06:25:26.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.863+0000 7f41e568c700 1 -- 172.21.15.3:0/3320625198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41e00cfd80 msgr2=0x7f41e00b85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:26.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.863+0000 7f41e568c700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41e00cfd80 0x7f41e00b85c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f41e006f190 tx=0x7f41d0004770 comp rx=0 tx=0).stop 2023-03-03T06:25:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.863+0000 7f41e568c700 1 -- 172.21.15.3:0/3320625198 shutdown_connections 2023-03-03T06:25:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.863+0000 7f41e568c700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f41cc064f10 0x7f41cc0673c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.863+0000 7f41e568c700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f41e015d6e0 0x7f41e00cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.863+0000 7f41e568c700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41e0156ba0 0x7f41e00c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.863+0000 7f41e568c700 1 --2- 172.21.15.3:0/3320625198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41e00cfd80 0x7f41e00b85c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.863+0000 7f41e568c700 1 -- 172.21.15.3:0/3320625198 >> 172.21.15.3:0/3320625198 conn(0x7f41e006e5c0 msgr2=0x7f41e015ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:26.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.867+0000 7f41e568c700 1 -- 172.21.15.3:0/3320625198 shutdown_connections 2023-03-03T06:25:26.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:26.867+0000 7f41e568c700 1 -- 172.21.15.3:0/3320625198 wait complete. 2023-03-03T06:25:26.905 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:25:27.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.107+0000 7fb71359e700 1 Processor -- start 2023-03-03T06:25:27.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.107+0000 7fb71359e700 1 -- start start 2023-03-03T06:25:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.111+0000 7fb71359e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb714156ba0 0x7fb714071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.111+0000 7fb71359e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7140cf3b0 con 0x7fb71415d6e0 2023-03-03T06:25:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.111+0000 7fb71359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb714156f70 con 0x7fb7140d0b30 2023-03-03T06:25:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.111+0000 7fb71359e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7140cd320 con 0x7fb714156ba0 2023-03-03T06:25:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.111+0000 7fb71259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb714156ba0 0x7fb714071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.115+0000 7fb711d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb71415d6e0 0x7fb7140cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35928/0 (socket says 172.21.15.3:35928) 2023-03-03T06:25:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.115+0000 7fb711d9b700 1 -- 172.21.15.3:0/2114513313 learned_addr learned my addr 172.21.15.3:0/2114513313 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.115+0000 7fb71159a700 1 -- 172.21.15.3:0/2114513313 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502628571 0 0) 0x7fb7140cf3b0 con 0x7fb71415d6e0 2023-03-03T06:25:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.115+0000 7fb71159a700 1 -- 172.21.15.3:0/2114513313 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6fc0035d0 con 0x7fb71415d6e0 2023-03-03T06:25:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.115+0000 7fb71259c700 1 -- 172.21.15.3:0/2114513313 >> v1:172.21.15.32:6789/0 conn(0x7fb7140d0b30 legacy=0x7fb7140b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.115+0000 7fb71259c700 1 -- 172.21.15.3:0/2114513313 >> v1:172.21.15.3:6789/0 conn(0x7fb71415d6e0 legacy=0x7fb7140cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.115+0000 7fb71259c700 1 -- 172.21.15.3:0/2114513313 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7140cd600 con 0x7fb714156ba0 2023-03-03T06:25:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.115+0000 7fb71259c700 1 --2- 172.21.15.3:0/2114513313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb714156ba0 0x7fb714071280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb704009850 tx=0x7fb704009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938c90fb94d5817 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.115+0000 7fb71159a700 1 -- 172.21.15.3:0/2114513313 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704011830 con 0x7fb714156ba0 2023-03-03T06:25:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.115+0000 7fb71159a700 1 -- 172.21.15.3:0/2114513313 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb704011990 con 0x7fb714156ba0 2023-03-03T06:25:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.115+0000 7fb71159a700 1 -- 172.21.15.3:0/2114513313 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704011d00 con 0x7fb714156ba0 2023-03-03T06:25:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.115+0000 7fb71359e700 1 -- 172.21.15.3:0/2114513313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb714156ba0 msgr2=0x7fb714071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.115+0000 7fb71359e700 1 --2- 172.21.15.3:0/2114513313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb714156ba0 0x7fb714071280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb704009850 tx=0x7fb704009b60 comp rx=0 tx=0).stop 2023-03-03T06:25:27.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.119+0000 7fb71359e700 1 -- 172.21.15.3:0/2114513313 shutdown_connections 2023-03-03T06:25:27.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.119+0000 7fb71359e700 1 --2- 172.21.15.3:0/2114513313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb714156ba0 0x7fb714071280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.119+0000 7fb71359e700 1 -- 172.21.15.3:0/2114513313 >> 172.21.15.3:0/2114513313 conn(0x7fb71406e5c0 msgr2=0x7fb71406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:27.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.119+0000 7fb71359e700 1 -- 172.21.15.3:0/2114513313 shutdown_connections 2023-03-03T06:25:27.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.119+0000 7fb71359e700 1 -- 172.21.15.3:0/2114513313 wait complete. 2023-03-03T06:25:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.119+0000 7fb71359e700 1 Processor -- start 2023-03-03T06:25:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.119+0000 7fb71359e700 1 -- start start 2023-03-03T06:25:27.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb71359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7140d0b30 0x7fb71414f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:27.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb71359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb714156ba0 0x7fb714154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:27.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb71359e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb71415d6e0 0x7fb71415f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:27.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb71359e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7141582f0 con 0x7fb714156ba0 2023-03-03T06:25:27.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb71359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71414fcf0 con 0x7fb7140d0b30 2023-03-03T06:25:27.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb71359e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7141576e0 con 0x7fb71415d6e0 2023-03-03T06:25:27.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb711d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb714156ba0 0x7fb714154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:27.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb711d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb714156ba0 0x7fb714154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44486/0 (socket says 172.21.15.3:44486) 2023-03-03T06:25:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb711d9b700 1 -- 172.21.15.3:0/3470726003 learned_addr learned my addr 172.21.15.3:0/3470726003 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb71259c700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7140d0b30 0x7fb71414f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb711d9b700 1 -- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb71415d6e0 msgr2=0x7fb71415f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb711d9b700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb71415d6e0 0x7fb71415f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb711d9b700 1 -- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7140d0b30 msgr2=0x7fb71414f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb711d9b700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7140d0b30 0x7fb71414f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb711d9b700 1 -- 172.21.15.3:0/3470726003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7040094b0 con 0x7fb714156ba0 2023-03-03T06:25:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb71259c700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7140d0b30 0x7fb71414f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:25:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb711d9b700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb714156ba0 0x7fb714154020 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c4d0 tx=0x7fb70800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb702ffd700 1 -- 172.21.15.3:0/3470726003 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708005d90 con 0x7fb714156ba0 2023-03-03T06:25:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb71359e700 1 -- 172.21.15.3:0/3470726003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7140717c0 con 0x7fb714156ba0 2023-03-03T06:25:27.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb702ffd700 1 -- 172.21.15.3:0/3470726003 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7080070d0 con 0x7fb714156ba0 2023-03-03T06:25:27.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb702ffd700 1 -- 172.21.15.3:0/3470726003 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708007480 con 0x7fb714156ba0 2023-03-03T06:25:27.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.123+0000 7fb71359e700 1 -- 172.21.15.3:0/3470726003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7140cd8e0 con 0x7fb714156ba0 2023-03-03T06:25:27.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.127+0000 7fb71359e700 1 -- 172.21.15.3:0/3470726003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7140cff50 con 0x7fb714156ba0 2023-03-03T06:25:27.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.127+0000 7fb702ffd700 1 -- 172.21.15.3:0/3470726003 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb708019030 con 0x7fb714156ba0 2023-03-03T06:25:27.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.131+0000 7fb702ffd700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb6f4064ee0 0x7fb6f4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:27.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.131+0000 7fb702ffd700 1 -- 172.21.15.3:0/3470726003 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb7080816f0 con 0x7fb714156ba0 2023-03-03T06:25:27.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.131+0000 7fb71259c700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb6f4064ee0 0x7fb6f4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:27.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.131+0000 7fb71259c700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb6f4064ee0 0x7fb6f4067390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb704000c00 tx=0x7fb704008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.139+0000 7fb702ffd700 1 -- 172.21.15.3:0/3470726003 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb70804f3d0 con 0x7fb714156ba0 2023-03-03T06:25:27.188 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:25:27.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:25:27.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.323+0000 7fb71359e700 1 -- 172.21.15.3:0/3470726003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb714073e90 con 0x7fb714156ba0 2023-03-03T06:25:27.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.327+0000 7fb702ffd700 1 -- 172.21.15.3:0/3470726003 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb7080529f0 con 0x7fb714156ba0 2023-03-03T06:25:27.330 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:25:27.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.327+0000 7fb71359e700 1 -- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb6f4064ee0 msgr2=0x7fb6f4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:27.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.327+0000 7fb71359e700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb6f4064ee0 0x7fb6f4067390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb704000c00 tx=0x7fb704008030 comp rx=0 tx=0).stop 2023-03-03T06:25:27.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.327+0000 7fb71359e700 1 -- 172.21.15.3:0/3470726003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb714156ba0 msgr2=0x7fb714154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:27.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.327+0000 7fb71359e700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb714156ba0 0x7fb714154020 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c4d0 tx=0x7fb70800c890 comp rx=0 tx=0).stop 2023-03-03T06:25:27.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.331+0000 7fb71359e700 1 -- 172.21.15.3:0/3470726003 shutdown_connections 2023-03-03T06:25:27.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.331+0000 7fb71359e700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb6f4064ee0 0x7fb6f4067390 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.331+0000 7fb71359e700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb71415d6e0 0x7fb71415f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.331+0000 7fb71359e700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb714156ba0 0x7fb714154020 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.331+0000 7fb71359e700 1 --2- 172.21.15.3:0/3470726003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7140d0b30 0x7fb71414f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.331+0000 7fb71359e700 1 -- 172.21.15.3:0/3470726003 >> 172.21.15.3:0/3470726003 conn(0x7fb71406e5c0 msgr2=0x7fb7140b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:27.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.331+0000 7fb71359e700 1 -- 172.21.15.3:0/3470726003 shutdown_connections 2023-03-03T06:25:27.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.331+0000 7fb71359e700 1 -- 172.21.15.3:0/3470726003 wait complete. 2023-03-03T06:25:27.369 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-03T06:25:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.571+0000 7f9b38dcb700 1 Processor -- start 2023-03-03T06:25:27.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.571+0000 7f9b38dcb700 1 -- start start 2023-03-03T06:25:27.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b38dcb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9b3415d6e0 0x7f9b340cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:27.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b38dcb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9b340cfd80 0x7f9b340d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b38dcb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b340cf3b0 con 0x7f9b34156ba0 2023-03-03T06:25:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b38dcb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b34156f70 con 0x7f9b340cfd80 2023-03-03T06:25:27.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b38dcb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b340cd320 con 0x7f9b3415d6e0 2023-03-03T06:25:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b337fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9b34156ba0 0x7f9b34071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35944/0 (socket says 172.21.15.3:35944) 2023-03-03T06:25:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b337fe700 1 -- 172.21.15.3:0/3352019704 learned_addr learned my addr 172.21.15.3:0/3352019704 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b33fff700 1 --2- 172.21.15.3:0/3352019704 >> v2:172.21.15.3:3300/0 conn(0x7f9b340cfd80 0x7f9b340d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b32ffd700 1 --2- 172.21.15.3:0/3352019704 >> v2:172.21.15.32:3300/0 conn(0x7f9b3415d6e0 0x7f9b340cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:27.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b33fff700 1 -- 172.21.15.3:0/3352019704 >> v2:172.21.15.32:3300/0 conn(0x7f9b3415d6e0 msgr2=0x7f9b340cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b33fff700 1 --2- 172.21.15.3:0/3352019704 >> v2:172.21.15.32:3300/0 conn(0x7f9b3415d6e0 0x7f9b340cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b33fff700 1 -- 172.21.15.3:0/3352019704 >> v1:172.21.15.3:6789/0 conn(0x7f9b34156ba0 legacy=0x7f9b34071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b33fff700 1 -- 172.21.15.3:0/3352019704 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b340cd600 con 0x7f9b340cfd80 2023-03-03T06:25:27.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b33fff700 1 --2- 172.21.15.3:0/3352019704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b340cfd80 0x7f9b340d0b30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9b340b1ca0 tx=0x7f9b2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24da961881fe95af server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b327fc700 1 -- 172.21.15.3:0/3352019704 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c00e070 con 0x7f9b340cfd80 2023-03-03T06:25:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b327fc700 1 -- 172.21.15.3:0/3352019704 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c00bbd0 con 0x7f9b340cfd80 2023-03-03T06:25:27.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.575+0000 7f9b327fc700 1 -- 172.21.15.3:0/3352019704 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c012430 con 0x7f9b340cfd80 2023-03-03T06:25:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.579+0000 7f9b38dcb700 1 -- 172.21.15.3:0/3352019704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b340cfd80 msgr2=0x7f9b340d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.579+0000 7f9b38dcb700 1 --2- 172.21.15.3:0/3352019704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b340cfd80 0x7f9b340d0b30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9b340b1ca0 tx=0x7f9b2c00b330 comp rx=0 tx=0).stop 2023-03-03T06:25:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.579+0000 7f9b38dcb700 1 -- 172.21.15.3:0/3352019704 shutdown_connections 2023-03-03T06:25:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.579+0000 7f9b38dcb700 1 --2- 172.21.15.3:0/3352019704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b340cfd80 0x7f9b340d0b30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.579+0000 7f9b38dcb700 1 --2- 172.21.15.3:0/3352019704 >> v2:172.21.15.32:3300/0 conn(0x7f9b3415d6e0 0x7f9b340cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.579+0000 7f9b38dcb700 1 -- 172.21.15.3:0/3352019704 >> 172.21.15.3:0/3352019704 conn(0x7f9b3406e5c0 msgr2=0x7f9b3406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.579+0000 7f9b38dcb700 1 -- 172.21.15.3:0/3352019704 shutdown_connections 2023-03-03T06:25:27.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.579+0000 7f9b38dcb700 1 -- 172.21.15.3:0/3352019704 wait complete. 2023-03-03T06:25:27.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.583+0000 7f9b38dcb700 1 Processor -- start 2023-03-03T06:25:27.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.583+0000 7f9b38dcb700 1 -- start start 2023-03-03T06:25:27.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.583+0000 7f9b38dcb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b340cfd80 0x7f9b340b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.583+0000 7f9b38dcb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b34156ba0 0x7f9b340c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.583+0000 7f9b38dcb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b3415d6e0 0x7f9b340c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.583+0000 7f9b38dcb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b340c8000 con 0x7f9b3415d6e0 2023-03-03T06:25:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.583+0000 7f9b38dcb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b340c82e0 con 0x7f9b34156ba0 2023-03-03T06:25:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.583+0000 7f9b38dcb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b340c85c0 con 0x7f9b340cfd80 2023-03-03T06:25:27.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.583+0000 7f9b33fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b3415d6e0 0x7f9b340c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:27.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.583+0000 7f9b337fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b340cfd80 0x7f9b340b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:27.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.583+0000 7f9b33fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b3415d6e0 0x7f9b340c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44500/0 (socket says 172.21.15.3:44500) 2023-03-03T06:25:27.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.583+0000 7f9b33fff700 1 -- 172.21.15.3:0/859374279 learned_addr learned my addr 172.21.15.3:0/859374279 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:27.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.583+0000 7f9b32ffd700 1 --2- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b34156ba0 0x7f9b340c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:27.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.587+0000 7f9b32ffd700 1 -- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b340cfd80 msgr2=0x7f9b340b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:27.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.587+0000 7f9b32ffd700 1 --2- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b340cfd80 0x7f9b340b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.587+0000 7f9b32ffd700 1 -- 172.21.15.3:0/859374279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b3415d6e0 msgr2=0x7f9b340c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:27.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.587+0000 7f9b32ffd700 1 --2- 172.21.15.3:0/859374279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b3415d6e0 0x7f9b340c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.587+0000 7f9b32ffd700 1 -- 172.21.15.3:0/859374279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c009a70 con 0x7f9b34156ba0 2023-03-03T06:25:27.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.587+0000 7f9b32ffd700 1 --2- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b34156ba0 0x7f9b340c90a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800d5f0 tx=0x7f9b2800d900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:27.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.587+0000 7f9b1bfff700 1 -- 172.21.15.3:0/859374279 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b280125f0 con 0x7f9b34156ba0 2023-03-03T06:25:27.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.587+0000 7f9b1bfff700 1 -- 172.21.15.3:0/859374279 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b28012750 con 0x7f9b34156ba0 2023-03-03T06:25:27.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.587+0000 7f9b1bfff700 1 -- 172.21.15.3:0/859374279 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28012b00 con 0x7f9b34156ba0 2023-03-03T06:25:27.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.587+0000 7f9b38dcb700 1 -- 172.21.15.3:0/859374279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b340cef40 con 0x7f9b34156ba0 2023-03-03T06:25:27.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.587+0000 7f9b38dcb700 1 -- 172.21.15.3:0/859374279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b340cd8e0 con 0x7f9b34156ba0 2023-03-03T06:25:27.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.591+0000 7f9b38dcb700 1 -- 172.21.15.3:0/859374279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b340cf0d0 con 0x7f9b34156ba0 2023-03-03T06:25:27.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.591+0000 7f9b1bfff700 1 -- 172.21.15.3:0/859374279 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9b28012c60 con 0x7f9b34156ba0 2023-03-03T06:25:27.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.595+0000 7f9b1bfff700 1 --2- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9b20064ee0 0x7f9b20067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:27.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.595+0000 7f9b1bfff700 1 -- 172.21.15.3:0/859374279 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9b280823b0 con 0x7f9b34156ba0 2023-03-03T06:25:27.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.595+0000 7f9b337fe700 1 --2- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9b20064ee0 0x7f9b20067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:27.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.595+0000 7f9b337fe700 1 --2- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9b20064ee0 0x7f9b20067390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9b24005930 tx=0x7f9b24009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:27.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.603+0000 7f9b1bfff700 1 -- 172.21.15.3:0/859374279 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9b28050170 con 0x7f9b34156ba0 2023-03-03T06:25:27.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.779+0000 7f9b38dcb700 1 -- 172.21.15.3:0/859374279 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b34073e90 con 0x7f9b34156ba0 2023-03-03T06:25:27.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.779+0000 7f9b1bfff700 1 -- 172.21.15.3:0/859374279 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b28053790 con 0x7f9b34156ba0 2023-03-03T06:25:27.785 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:25:27.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.783+0000 7f9b38dcb700 1 -- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9b20064ee0 msgr2=0x7f9b20067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:27.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.783+0000 7f9b38dcb700 1 --2- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9b20064ee0 0x7f9b20067390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9b24005930 tx=0x7f9b24009040 comp rx=0 tx=0).stop 2023-03-03T06:25:27.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.783+0000 7f9b38dcb700 1 -- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b34156ba0 msgr2=0x7f9b340c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:27.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.783+0000 7f9b38dcb700 1 --2- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b34156ba0 0x7f9b340c90a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800d5f0 tx=0x7f9b2800d900 comp rx=0 tx=0).stop 2023-03-03T06:25:27.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.783+0000 7f9b38dcb700 1 -- 172.21.15.3:0/859374279 shutdown_connections 2023-03-03T06:25:27.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.783+0000 7f9b38dcb700 1 --2- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9b20064ee0 0x7f9b20067390 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.783+0000 7f9b38dcb700 1 --2- 172.21.15.3:0/859374279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b3415d6e0 0x7f9b340c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.783+0000 7f9b38dcb700 1 --2- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b34156ba0 0x7f9b340c90a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.783+0000 7f9b38dcb700 1 --2- 172.21.15.3:0/859374279 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9b340cfd80 0x7f9b340b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:27.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.783+0000 7f9b38dcb700 1 -- 172.21.15.3:0/859374279 >> 172.21.15.3:0/859374279 conn(0x7f9b3406e5c0 msgr2=0x7f9b34071280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:27.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.787+0000 7f9b38dcb700 1 -- 172.21.15.3:0/859374279 shutdown_connections 2023-03-03T06:25:27.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:27.787+0000 7f9b38dcb700 1 -- 172.21.15.3:0/859374279 wait complete. 2023-03-03T06:25:27.824 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-03T06:25:28.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.035+0000 7f1e3ca24700 1 Processor -- start 2023-03-03T06:25:28.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.039+0000 7f1e3ca24700 1 -- start start 2023-03-03T06:25:28.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e3ca24700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e381591d0 0x7f1e380cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e3ca24700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e3806d490 0x7f1e380b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e3ca24700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e380ce810 con 0x7f1e380b07c0 2023-03-03T06:25:28.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e3ca24700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e381595a0 con 0x7f1e381591d0 2023-03-03T06:25:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e3ca24700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380b1180 con 0x7f1e3806d490 2023-03-03T06:25:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e36d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e381591d0 0x7f1e380cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e36d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e381591d0 0x7f1e380cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:51018/0 (socket says 172.21.15.3:51018) 2023-03-03T06:25:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e36d9d700 1 -- 172.21.15.3:0/2549963996 learned_addr learned my addr 172.21.15.3:0/2549963996 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:28.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e3759e700 1 --2- 172.21.15.3:0/2549963996 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e3806d490 0x7f1e380b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e36d9d700 1 -- 172.21.15.3:0/2549963996 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e3806d490 msgr2=0x7f1e380b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:28.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e36d9d700 1 --2- 172.21.15.3:0/2549963996 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e3806d490 0x7f1e380b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e36d9d700 1 -- 172.21.15.3:0/2549963996 >> v1:172.21.15.32:6789/0 conn(0x7f1e380b07c0 legacy=0x7f1e3815cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e36d9d700 1 -- 172.21.15.3:0/2549963996 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e380ceaf0 con 0x7f1e381591d0 2023-03-03T06:25:28.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.043+0000 7f1e36d9d700 1 --2- 172.21.15.3:0/2549963996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e381591d0 0x7f1e380cd6d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009910 tx=0x7f1e28009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c63ef92107b34261 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.047+0000 7f1e35d9b700 1 -- 172.21.15.3:0/2549963996 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28011830 con 0x7f1e381591d0 2023-03-03T06:25:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.047+0000 7f1e35d9b700 1 -- 172.21.15.3:0/2549963996 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e28011990 con 0x7f1e381591d0 2023-03-03T06:25:28.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.047+0000 7f1e35d9b700 1 -- 172.21.15.3:0/2549963996 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28011d40 con 0x7f1e381591d0 2023-03-03T06:25:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.047+0000 7f1e3ca24700 1 -- 172.21.15.3:0/2549963996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e381591d0 msgr2=0x7f1e380cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.047+0000 7f1e3ca24700 1 --2- 172.21.15.3:0/2549963996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e381591d0 0x7f1e380cd6d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009910 tx=0x7f1e28009c20 comp rx=0 tx=0).stop 2023-03-03T06:25:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.047+0000 7f1e3ca24700 1 -- 172.21.15.3:0/2549963996 shutdown_connections 2023-03-03T06:25:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.047+0000 7f1e3ca24700 1 --2- 172.21.15.3:0/2549963996 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e3806d490 0x7f1e380b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.047+0000 7f1e3ca24700 1 --2- 172.21.15.3:0/2549963996 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e381591d0 0x7f1e380cd6d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.047+0000 7f1e3ca24700 1 -- 172.21.15.3:0/2549963996 >> 172.21.15.3:0/2549963996 conn(0x7f1e380607c0 msgr2=0x7f1e38060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:28.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.047+0000 7f1e3ca24700 1 -- 172.21.15.3:0/2549963996 shutdown_connections 2023-03-03T06:25:28.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.047+0000 7f1e3ca24700 1 -- 172.21.15.3:0/2549963996 wait complete. 2023-03-03T06:25:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3ca24700 1 Processor -- start 2023-03-03T06:25:28.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3ca24700 1 -- start start 2023-03-03T06:25:28.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3ca24700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e3806d490 0x7f1e380c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3ca24700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e380b07c0 0x7f1e380b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3ca24700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e381591d0 0x7f1e381f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3ca24700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e381526b0 con 0x7f1e380b07c0 2023-03-03T06:25:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3ca24700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380b3e50 con 0x7f1e3806d490 2023-03-03T06:25:28.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3ca24700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380b4130 con 0x7f1e381591d0 2023-03-03T06:25:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e380b07c0 0x7f1e380b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e380b07c0 0x7f1e380b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44516/0 (socket says 172.21.15.3:44516) 2023-03-03T06:25:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3659c700 1 -- 172.21.15.3:0/3160356942 learned_addr learned my addr 172.21.15.3:0/3160356942 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e36d9d700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e3806d490 0x7f1e380c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3759e700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e381591d0 0x7f1e381f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3659c700 1 -- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e381591d0 msgr2=0x7f1e381f12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3659c700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e381591d0 0x7f1e381f12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3659c700 1 -- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e3806d490 msgr2=0x7f1e380c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.051+0000 7f1e3659c700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e3806d490 0x7f1e380c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.055+0000 7f1e3659c700 1 -- 172.21.15.3:0/3160356942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e28009570 con 0x7f1e380b07c0 2023-03-03T06:25:28.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.055+0000 7f1e36d9d700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e3806d490 0x7f1e380c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:28.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.055+0000 7f1e3659c700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e380b07c0 0x7f1e380b16a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00c500 tx=0x7f1e2c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.055+0000 7f1e1f7fe700 1 -- 172.21.15.3:0/3160356942 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0052d0 con 0x7f1e380b07c0 2023-03-03T06:25:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.055+0000 7f1e1f7fe700 1 -- 172.21.15.3:0/3160356942 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c005430 con 0x7f1e380b07c0 2023-03-03T06:25:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.055+0000 7f1e3ca24700 1 -- 172.21.15.3:0/3160356942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e380cf0b0 con 0x7f1e380b07c0 2023-03-03T06:25:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.055+0000 7f1e1f7fe700 1 -- 172.21.15.3:0/3160356942 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0055a0 con 0x7f1e380b07c0 2023-03-03T06:25:28.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.055+0000 7f1e3ca24700 1 -- 172.21.15.3:0/3160356942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e380cedd0 con 0x7f1e380b07c0 2023-03-03T06:25:28.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.059+0000 7f1e3ca24700 1 -- 172.21.15.3:0/3160356942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e380cf390 con 0x7f1e380b07c0 2023-03-03T06:25:28.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.059+0000 7f1e1f7fe700 1 -- 172.21.15.3:0/3160356942 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1e2c019030 con 0x7f1e380b07c0 2023-03-03T06:25:28.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.063+0000 7f1e1f7fe700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1e24064ee0 0x7f1e24067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.063+0000 7f1e1f7fe700 1 -- 172.21.15.3:0/3160356942 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1e2c081550 con 0x7f1e380b07c0 2023-03-03T06:25:28.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.063+0000 7f1e36d9d700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1e24064ee0 0x7f1e24067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.063+0000 7f1e36d9d700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1e24064ee0 0x7f1e24067390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e28000c00 tx=0x7f1e28008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:28.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.067+0000 7f1e1f7fe700 1 -- 172.21.15.3:0/3160356942 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1e2c04f230 con 0x7f1e380b07c0 2023-03-03T06:25:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.247+0000 7f1e3ca24700 1 -- 172.21.15.3:0/3160356942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e38073e90 con 0x7f1e380b07c0 2023-03-03T06:25:28.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.247+0000 7f1e1f7fe700 1 -- 172.21.15.3:0/3160356942 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e2c052850 con 0x7f1e380b07c0 2023-03-03T06:25:28.252 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-03-03T06:25:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.251+0000 7f1e3ca24700 1 -- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1e24064ee0 msgr2=0x7f1e24067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.251+0000 7f1e3ca24700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1e24064ee0 0x7f1e24067390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e28000c00 tx=0x7f1e28008030 comp rx=0 tx=0).stop 2023-03-03T06:25:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.251+0000 7f1e3ca24700 1 -- 172.21.15.3:0/3160356942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e380b07c0 msgr2=0x7f1e380b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.251+0000 7f1e3ca24700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e380b07c0 0x7f1e380b16a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00c500 tx=0x7f1e2c00c810 comp rx=0 tx=0).stop 2023-03-03T06:25:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.251+0000 7f1e3ca24700 1 -- 172.21.15.3:0/3160356942 shutdown_connections 2023-03-03T06:25:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.251+0000 7f1e3ca24700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1e24064ee0 0x7f1e24067390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.251+0000 7f1e3ca24700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e381591d0 0x7f1e381f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.251+0000 7f1e3ca24700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e380b07c0 0x7f1e380b16a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.251+0000 7f1e3ca24700 1 --2- 172.21.15.3:0/3160356942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e3806d490 0x7f1e380c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.251+0000 7f1e3ca24700 1 -- 172.21.15.3:0/3160356942 >> 172.21.15.3:0/3160356942 conn(0x7f1e380607c0 msgr2=0x7f1e380aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.251+0000 7f1e3ca24700 1 -- 172.21.15.3:0/3160356942 shutdown_connections 2023-03-03T06:25:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.251+0000 7f1e3ca24700 1 -- 172.21.15.3:0/3160356942 wait complete. 2023-03-03T06:25:28.290 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-03T06:25:28.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.495+0000 7f83425a8700 1 Processor -- start 2023-03-03T06:25:28.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.495+0000 7f83425a8700 1 -- start start 2023-03-03T06:25:28.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.499+0000 7f83425a8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f833c15fb60 0x7f833c160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.499+0000 7f83425a8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f833c06d7b0 0x7f833c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.499+0000 7f83425a8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c0ccf20 con 0x7f833c0c95c0 2023-03-03T06:25:28.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.499+0000 7f83425a8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833c15ff30 con 0x7f833c06d7b0 2023-03-03T06:25:28.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.499+0000 7f83425a8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833c0cd200 con 0x7f833c15fb60 2023-03-03T06:25:28.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.499+0000 7f8341da7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f833c06d7b0 0x7f833c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.499+0000 7f83415a6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f833c15fb60 0x7f833c160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.499+0000 7f8341da7700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f833c06d7b0 0x7f833c071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44526/0 (socket says 172.21.15.3:44526) 2023-03-03T06:25:28.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.499+0000 7f8341da7700 1 -- 172.21.15.3:0/965750485 learned_addr learned my addr 172.21.15.3:0/965750485 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.499+0000 7f8341da7700 1 -- 172.21.15.3:0/965750485 >> v2:172.21.15.32:3300/0 conn(0x7f833c15fb60 msgr2=0x7f833c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.499+0000 7f8341da7700 1 --2- 172.21.15.3:0/965750485 >> v2:172.21.15.32:3300/0 conn(0x7f833c15fb60 0x7f833c160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.499+0000 7f8341da7700 1 -- 172.21.15.3:0/965750485 >> v1:172.21.15.32:6789/0 conn(0x7f833c0c95c0 legacy=0x7f833c06d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.499+0000 7f8341da7700 1 -- 172.21.15.3:0/965750485 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c0cd4e0 con 0x7f833c06d7b0 2023-03-03T06:25:28.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.503+0000 7f8341da7700 1 --2- 172.21.15.3:0/965750485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f833c06d7b0 0x7f833c071230 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f833c06c7a0 tx=0x7f833400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39fddd01abb49bf server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.503+0000 7f8333fff700 1 -- 172.21.15.3:0/965750485 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400e070 con 0x7f833c06d7b0 2023-03-03T06:25:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.503+0000 7f8333fff700 1 -- 172.21.15.3:0/965750485 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833400b910 con 0x7f833c06d7b0 2023-03-03T06:25:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.503+0000 7f8333fff700 1 -- 172.21.15.3:0/965750485 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400bcc0 con 0x7f833c06d7b0 2023-03-03T06:25:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.503+0000 7f83425a8700 1 -- 172.21.15.3:0/965750485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f833c06d7b0 msgr2=0x7f833c071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.503+0000 7f83425a8700 1 --2- 172.21.15.3:0/965750485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f833c06d7b0 0x7f833c071230 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f833c06c7a0 tx=0x7f833400b330 comp rx=0 tx=0).stop 2023-03-03T06:25:28.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.503+0000 7f83425a8700 1 -- 172.21.15.3:0/965750485 shutdown_connections 2023-03-03T06:25:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.503+0000 7f83425a8700 1 --2- 172.21.15.3:0/965750485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f833c06d7b0 0x7f833c071230 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.503+0000 7f83425a8700 1 --2- 172.21.15.3:0/965750485 >> v2:172.21.15.32:3300/0 conn(0x7f833c15fb60 0x7f833c160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.503+0000 7f83425a8700 1 -- 172.21.15.3:0/965750485 >> 172.21.15.3:0/965750485 conn(0x7f833c0cfc90 msgr2=0x7f833c0cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.503+0000 7f83425a8700 1 -- 172.21.15.3:0/965750485 shutdown_connections 2023-03-03T06:25:28.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.503+0000 7f83425a8700 1 -- 172.21.15.3:0/965750485 wait complete. 2023-03-03T06:25:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.507+0000 7f83425a8700 1 Processor -- start 2023-03-03T06:25:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.507+0000 7f83425a8700 1 -- start start 2023-03-03T06:25:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f83425a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f833c06d7b0 0x7f833c1f9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f83425a8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f833c0c95c0 0x7f833c1f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f83425a8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f833c15fb60 0x7f833c1f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f83425a8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c0c56b0 con 0x7f833c15fb60 2023-03-03T06:25:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f83425a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c0c32a0 con 0x7f833c06d7b0 2023-03-03T06:25:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f83425a8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833c0c3580 con 0x7f833c0c95c0 2023-03-03T06:25:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f8341da7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f833c15fb60 0x7f833c1f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f8341da7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f833c15fb60 0x7f833c1f68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44536/0 (socket says 172.21.15.3:44536) 2023-03-03T06:25:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f8341da7700 1 -- 172.21.15.3:0/1814199931 learned_addr learned my addr 172.21.15.3:0/1814199931 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f83415a6700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f833c06d7b0 0x7f833c1f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f8340da5700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f833c0c95c0 0x7f833c1f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f8341da7700 1 -- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f833c0c95c0 msgr2=0x7f833c1f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f8341da7700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f833c0c95c0 0x7f833c1f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f8341da7700 1 -- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f833c06d7b0 msgr2=0x7f833c1f9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f8341da7700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f833c06d7b0 0x7f833c1f9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f8341da7700 1 -- 172.21.15.3:0/1814199931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8334009b30 con 0x7f833c15fb60 2023-03-03T06:25:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f8341da7700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f833c15fb60 0x7f833c1f68d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8334012000 tx=0x7f8334004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f8331ffb700 1 -- 172.21.15.3:0/1814199931 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83340099b0 con 0x7f833c15fb60 2023-03-03T06:25:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f8331ffb700 1 -- 172.21.15.3:0/1814199931 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83340092e0 con 0x7f833c15fb60 2023-03-03T06:25:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f8331ffb700 1 -- 172.21.15.3:0/1814199931 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833401c640 con 0x7f833c15fb60 2023-03-03T06:25:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f83425a8700 1 -- 172.21.15.3:0/1814199931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833c06cb80 con 0x7f833c15fb60 2023-03-03T06:25:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.511+0000 7f83425a8700 1 -- 172.21.15.3:0/1814199931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833c06c8a0 con 0x7f833c15fb60 2023-03-03T06:25:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.515+0000 7f83425a8700 1 -- 172.21.15.3:0/1814199931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833c161000 con 0x7f833c15fb60 2023-03-03T06:25:28.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.515+0000 7f8331ffb700 1 -- 172.21.15.3:0/1814199931 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8334022030 con 0x7f833c15fb60 2023-03-03T06:25:28.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.519+0000 7f8331ffb700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8320064ee0 0x7f8320067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.519+0000 7f8331ffb700 1 -- 172.21.15.3:0/1814199931 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8334083b50 con 0x7f833c15fb60 2023-03-03T06:25:28.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.519+0000 7f83415a6700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8320064ee0 0x7f8320067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.523+0000 7f83415a6700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8320064ee0 0x7f8320067390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8338009ae0 tx=0x7f8338008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:28.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.523+0000 7f8331ffb700 1 -- 172.21.15.3:0/1814199931 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f8334051910 con 0x7f833c15fb60 2023-03-03T06:25:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.695+0000 7f83425a8700 1 -- 172.21.15.3:0/1814199931 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f833c073e90 con 0x7f833c15fb60 2023-03-03T06:25:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.695+0000 7f8331ffb700 1 -- 172.21.15.3:0/1814199931 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8334054f30 con 0x7f833c15fb60 2023-03-03T06:25:28.702 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:25:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.703+0000 7f83425a8700 1 -- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8320064ee0 msgr2=0x7f8320067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.703+0000 7f83425a8700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8320064ee0 0x7f8320067390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8338009ae0 tx=0x7f8338008040 comp rx=0 tx=0).stop 2023-03-03T06:25:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.703+0000 7f83425a8700 1 -- 172.21.15.3:0/1814199931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f833c15fb60 msgr2=0x7f833c1f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.703+0000 7f83425a8700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f833c15fb60 0x7f833c1f68d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8334012000 tx=0x7f8334004af0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.703+0000 7f83425a8700 1 -- 172.21.15.3:0/1814199931 shutdown_connections 2023-03-03T06:25:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.703+0000 7f83425a8700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8320064ee0 0x7f8320067390 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.703+0000 7f83425a8700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f833c15fb60 0x7f833c1f68d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.703+0000 7f83425a8700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f833c0c95c0 0x7f833c1f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.703+0000 7f83425a8700 1 --2- 172.21.15.3:0/1814199931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f833c06d7b0 0x7f833c1f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.703+0000 7f83425a8700 1 -- 172.21.15.3:0/1814199931 >> 172.21.15.3:0/1814199931 conn(0x7f833c0cfc90 msgr2=0x7f833c0cb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:28.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.703+0000 7f83425a8700 1 -- 172.21.15.3:0/1814199931 shutdown_connections 2023-03-03T06:25:28.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.703+0000 7f83425a8700 1 -- 172.21.15.3:0/1814199931 wait complete. 2023-03-03T06:25:28.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-03T06:25:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.955+0000 7ffa7202d700 1 Processor -- start 2023-03-03T06:25:28.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.955+0000 7ffa7202d700 1 -- start start 2023-03-03T06:25:28.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.959+0000 7ffa7202d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa6c1608d0 0x7ffa6c0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.959+0000 7ffa7202d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6c160440 con 0x7ffa6c153670 2023-03-03T06:25:28.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.959+0000 7ffa7202d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6c0cd3e0 con 0x7ffa6c160d20 2023-03-03T06:25:28.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.959+0000 7ffa7202d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c0cd6c0 con 0x7ffa6c1608d0 2023-03-03T06:25:28.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.959+0000 7ffa7102b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffa6c153670 0x7ffa6c071350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35948/0 (socket says 172.21.15.3:35948) 2023-03-03T06:25:28.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.959+0000 7ffa7102b700 1 -- 172.21.15.3:0/1335729355 learned_addr learned my addr 172.21.15.3:0/1335729355 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:28.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.959+0000 7ffa7082a700 1 --2- 172.21.15.3:0/1335729355 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa6c1608d0 0x7ffa6c0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.963+0000 7ffa7082a700 1 -- 172.21.15.3:0/1335729355 >> v1:172.21.15.32:6789/0 conn(0x7ffa6c160d20 legacy=0x7ffa6c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.963+0000 7ffa63fff700 1 -- 172.21.15.3:0/1335729355 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600797769 0 0) 0x7ffa6c160440 con 0x7ffa6c153670 2023-03-03T06:25:28.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.963+0000 7ffa7082a700 1 -- 172.21.15.3:0/1335729355 >> v1:172.21.15.3:6789/0 conn(0x7ffa6c153670 legacy=0x7ffa6c071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.963+0000 7ffa7082a700 1 -- 172.21.15.3:0/1335729355 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c1600d0 con 0x7ffa6c1608d0 2023-03-03T06:25:28.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.963+0000 7ffa7082a700 1 --2- 172.21.15.3:0/1335729355 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa6c1608d0 0x7ffa6c0cf620 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009880 tx=0x7ffa5c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f6f77d5c08b982b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.963+0000 7ffa63fff700 1 -- 172.21.15.3:0/1335729355 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c011830 con 0x7ffa6c1608d0 2023-03-03T06:25:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.963+0000 7ffa63fff700 1 -- 172.21.15.3:0/1335729355 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c011990 con 0x7ffa6c1608d0 2023-03-03T06:25:28.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.963+0000 7ffa63fff700 1 -- 172.21.15.3:0/1335729355 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c011d20 con 0x7ffa6c1608d0 2023-03-03T06:25:28.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.963+0000 7ffa7202d700 1 -- 172.21.15.3:0/1335729355 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa6c1608d0 msgr2=0x7ffa6c0cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.963+0000 7ffa7202d700 1 --2- 172.21.15.3:0/1335729355 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa6c1608d0 0x7ffa6c0cf620 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009880 tx=0x7ffa5c009b90 comp rx=0 tx=0).stop 2023-03-03T06:25:28.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.963+0000 7ffa7202d700 1 -- 172.21.15.3:0/1335729355 shutdown_connections 2023-03-03T06:25:28.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.963+0000 7ffa7202d700 1 --2- 172.21.15.3:0/1335729355 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa6c1608d0 0x7ffa6c0cf620 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.963+0000 7ffa7202d700 1 -- 172.21.15.3:0/1335729355 >> 172.21.15.3:0/1335729355 conn(0x7ffa6c06e5c0 msgr2=0x7ffa6c06e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:28.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.967+0000 7ffa7202d700 1 -- 172.21.15.3:0/1335729355 shutdown_connections 2023-03-03T06:25:28.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.967+0000 7ffa7202d700 1 -- 172.21.15.3:0/1335729355 wait complete. 2023-03-03T06:25:28.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.971+0000 7ffa7202d700 1 Processor -- start 2023-03-03T06:25:28.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.971+0000 7ffa7202d700 1 -- start start 2023-03-03T06:25:28.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.971+0000 7ffa7202d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa6c153670 0x7ffa6c1f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.971+0000 7ffa7202d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa6c1608d0 0x7ffa6c0b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.971+0000 7ffa7202d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa6c160d20 0x7ffa6c0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.971+0000 7ffa7202d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c0b7b20 con 0x7ffa6c153670 2023-03-03T06:25:28.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.971+0000 7ffa7202d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c1f2290 con 0x7ffa6c160d20 2023-03-03T06:25:28.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.971+0000 7ffa7202d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c1f2570 con 0x7ffa6c1608d0 2023-03-03T06:25:28.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.971+0000 7ffa7102b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa6c153670 0x7ffa6c1f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.971+0000 7ffa7102b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa6c153670 0x7ffa6c1f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44540/0 (socket says 172.21.15.3:44540) 2023-03-03T06:25:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.971+0000 7ffa7102b700 1 -- 172.21.15.3:0/1776548220 learned_addr learned my addr 172.21.15.3:0/1776548220 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.971+0000 7ffa7182c700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa6c160d20 0x7ffa6c0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.971+0000 7ffa7082a700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa6c1608d0 0x7ffa6c0b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.975+0000 7ffa7102b700 1 -- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa6c1608d0 msgr2=0x7ffa6c0b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.975+0000 7ffa7102b700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa6c1608d0 0x7ffa6c0b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.975+0000 7ffa7102b700 1 -- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa6c160d20 msgr2=0x7ffa6c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.975+0000 7ffa7102b700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa6c160d20 0x7ffa6c0cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.975+0000 7ffa7102b700 1 -- 172.21.15.3:0/1776548220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5c0094b0 con 0x7ffa6c153670 2023-03-03T06:25:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.975+0000 7ffa7082a700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa6c1608d0 0x7ffa6c0b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.975+0000 7ffa7102b700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa6c153670 0x7ffa6c1f1c60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800cc80 tx=0x7ffa6800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.975+0000 7ffa61ffb700 1 -- 172.21.15.3:0/1776548220 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800ade0 con 0x7ffa6c153670 2023-03-03T06:25:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.975+0000 7ffa61ffb700 1 -- 172.21.15.3:0/1776548220 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6800ce80 con 0x7ffa6c153670 2023-03-03T06:25:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.975+0000 7ffa7202d700 1 -- 172.21.15.3:0/1776548220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6c0cdb30 con 0x7ffa6c153670 2023-03-03T06:25:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.975+0000 7ffa61ffb700 1 -- 172.21.15.3:0/1776548220 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68007310 con 0x7ffa6c153670 2023-03-03T06:25:28.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.975+0000 7ffa7202d700 1 -- 172.21.15.3:0/1776548220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6c0cd9a0 con 0x7ffa6c153670 2023-03-03T06:25:28.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.975+0000 7ffa7202d700 1 -- 172.21.15.3:0/1776548220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6c0cf200 con 0x7ffa6c153670 2023-03-03T06:25:28.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.979+0000 7ffa61ffb700 1 -- 172.21.15.3:0/1776548220 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ffa68010070 con 0x7ffa6c153670 2023-03-03T06:25:28.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.983+0000 7ffa61ffb700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ffa50064ee0 0x7ffa50067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:28.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.983+0000 7ffa61ffb700 1 -- 172.21.15.3:0/1776548220 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffa68081de0 con 0x7ffa6c153670 2023-03-03T06:25:28.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.983+0000 7ffa7082a700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ffa50064ee0 0x7ffa50067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:28.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.987+0000 7ffa7082a700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ffa50064ee0 0x7ffa50067390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0097d0 tx=0x7ffa5c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:28.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:28.991+0000 7ffa61ffb700 1 -- 172.21.15.3:0/1776548220 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ffa6804fb40 con 0x7ffa6c153670 2023-03-03T06:25:29.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.183+0000 7ffa7202d700 1 -- 172.21.15.3:0/1776548220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffa6c073e90 con 0x7ffa6c153670 2023-03-03T06:25:29.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.183+0000 7ffa61ffb700 1 -- 172.21.15.3:0/1776548220 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffa68018020 con 0x7ffa6c153670 2023-03-03T06:25:29.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.187+0000 7ffa7202d700 1 -- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ffa50064ee0 msgr2=0x7ffa50067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.187+0000 7ffa7202d700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ffa50064ee0 0x7ffa50067390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0097d0 tx=0x7ffa5c008030 comp rx=0 tx=0).stop 2023-03-03T06:25:29.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.187+0000 7ffa7202d700 1 -- 172.21.15.3:0/1776548220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa6c153670 msgr2=0x7ffa6c1f1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.187+0000 7ffa7202d700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa6c153670 0x7ffa6c1f1c60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800cc80 tx=0x7ffa6800a340 comp rx=0 tx=0).stop 2023-03-03T06:25:29.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.187+0000 7ffa7202d700 1 -- 172.21.15.3:0/1776548220 shutdown_connections 2023-03-03T06:25:29.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.187+0000 7ffa7202d700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ffa50064ee0 0x7ffa50067390 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.187+0000 7ffa7202d700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa6c160d20 0x7ffa6c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.187+0000 7ffa7202d700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa6c1608d0 0x7ffa6c0b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.187+0000 7ffa7202d700 1 --2- 172.21.15.3:0/1776548220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa6c153670 0x7ffa6c1f1c60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.187+0000 7ffa7202d700 1 -- 172.21.15.3:0/1776548220 >> 172.21.15.3:0/1776548220 conn(0x7ffa6c06e5c0 msgr2=0x7ffa6c14e210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:29.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.187+0000 7ffa7202d700 1 -- 172.21.15.3:0/1776548220 shutdown_connections 2023-03-03T06:25:29.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.187+0000 7ffa7202d700 1 -- 172.21.15.3:0/1776548220 wait complete. 2023-03-03T06:25:29.196 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2023-03-03T06:25:29.226 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:25:29.226 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:25:29.246 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:25:29.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.491+0000 7f591dd97700 1 Processor -- start 2023-03-03T06:25:29.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.491+0000 7f591dd97700 1 -- start start 2023-03-03T06:25:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.495+0000 7f591dd97700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59180cfd30 0x7f59180d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.495+0000 7f591dd97700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5918160440 con 0x7f59181608d0 2023-03-03T06:25:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.495+0000 7f591dd97700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59180cd3e0 con 0x7f5918153670 2023-03-03T06:25:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.495+0000 7f590ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f59181608d0 0x7f59180cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35952/0 (socket says 172.21.15.3:35952) 2023-03-03T06:25:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.495+0000 7f591dd97700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59180cd6c0 con 0x7f59180cfd30 2023-03-03T06:25:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.495+0000 7f590ffff700 1 -- 172.21.15.3:0/3452970247 learned_addr learned my addr 172.21.15.3:0/3452970247 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:29.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.495+0000 7f591d596700 1 --2- 172.21.15.3:0/3452970247 >> v2:172.21.15.3:3300/0 conn(0x7f59180cfd30 0x7f59180d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.495+0000 7f590f7fe700 1 -- 172.21.15.3:0/3452970247 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543415784 0 0) 0x7f5918160440 con 0x7f59181608d0 2023-03-03T06:25:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f590f7fe700 1 -- 172.21.15.3:0/3452970247 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59000035d0 con 0x7f59181608d0 2023-03-03T06:25:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f590f7fe700 1 -- 172.21.15.3:0/3452970247 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843656468 0 0) 0x7f59180cd3e0 con 0x7f5918153670 2023-03-03T06:25:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f590f7fe700 1 -- 172.21.15.3:0/3452970247 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5918160440 con 0x7f5918153670 2023-03-03T06:25:29.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f591d596700 1 -- 172.21.15.3:0/3452970247 >> v1:172.21.15.32:6789/0 conn(0x7f5918153670 legacy=0x7f5918071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f591d596700 1 -- 172.21.15.3:0/3452970247 >> v1:172.21.15.3:6789/0 conn(0x7f59181608d0 legacy=0x7f59180cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f591d596700 1 -- 172.21.15.3:0/3452970247 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59181600d0 con 0x7f59180cfd30 2023-03-03T06:25:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f591d596700 1 --2- 172.21.15.3:0/3452970247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59180cfd30 0x7f59180d0b10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5910009dc0 tx=0x7f591000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e5338effa6df9f server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:29.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f590f7fe700 1 -- 172.21.15.3:0/3452970247 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591000e070 con 0x7f59180cfd30 2023-03-03T06:25:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f590f7fe700 1 -- 172.21.15.3:0/3452970247 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591000bae0 con 0x7f59180cfd30 2023-03-03T06:25:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f590f7fe700 1 -- 172.21.15.3:0/3452970247 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591000be90 con 0x7f59180cfd30 2023-03-03T06:25:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f591dd97700 1 -- 172.21.15.3:0/3452970247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59180cfd30 msgr2=0x7f59180d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f591dd97700 1 --2- 172.21.15.3:0/3452970247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59180cfd30 0x7f59180d0b10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5910009dc0 tx=0x7f591000b330 comp rx=0 tx=0).stop 2023-03-03T06:25:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f591dd97700 1 -- 172.21.15.3:0/3452970247 shutdown_connections 2023-03-03T06:25:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f591dd97700 1 --2- 172.21.15.3:0/3452970247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59180cfd30 0x7f59180d0b10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f591dd97700 1 -- 172.21.15.3:0/3452970247 >> 172.21.15.3:0/3452970247 conn(0x7f591806e5c0 msgr2=0x7f591806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f591dd97700 1 -- 172.21.15.3:0/3452970247 shutdown_connections 2023-03-03T06:25:29.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.499+0000 7f591dd97700 1 -- 172.21.15.3:0/3452970247 wait complete. 2023-03-03T06:25:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.503+0000 7f591dd97700 1 Processor -- start 2023-03-03T06:25:29.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f591dd97700 1 -- start start 2023-03-03T06:25:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f591dd97700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59180cfd30 0x7f591814dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f591dd97700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918153670 0x7f59180c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f591dd97700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59181608d0 0x7f591814cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f591dd97700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59180ca6f0 con 0x7f5918153670 2023-03-03T06:25:29.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f591dd97700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59180ca9d0 con 0x7f59181608d0 2023-03-03T06:25:29.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f591dd97700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59180cacb0 con 0x7f59180cfd30 2023-03-03T06:25:29.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f590ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918153670 0x7f59180c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:29.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f590ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918153670 0x7f59180c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44568/0 (socket says 172.21.15.3:44568) 2023-03-03T06:25:29.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f590ffff700 1 -- 172.21.15.3:0/1032223517 learned_addr learned my addr 172.21.15.3:0/1032223517 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:29.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f591cd95700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59180cfd30 0x7f591814dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:29.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f591d596700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59181608d0 0x7f591814cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:29.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f590ffff700 1 -- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59180cfd30 msgr2=0x7f591814dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f590ffff700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59180cfd30 0x7f591814dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f590ffff700 1 -- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59181608d0 msgr2=0x7f591814cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f590ffff700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59181608d0 0x7f591814cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f590ffff700 1 -- 172.21.15.3:0/1032223517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5910009a70 con 0x7f5918153670 2023-03-03T06:25:29.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.507+0000 7f590ffff700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918153670 0x7f59180c9ce0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f590800c4d0 tx=0x7f590800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:29.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.511+0000 7f590d7fa700 1 -- 172.21.15.3:0/1032223517 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908005d90 con 0x7f5918153670 2023-03-03T06:25:29.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.511+0000 7f590d7fa700 1 -- 172.21.15.3:0/1032223517 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59080070d0 con 0x7f5918153670 2023-03-03T06:25:29.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.511+0000 7f591dd97700 1 -- 172.21.15.3:0/1032223517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59180cdb30 con 0x7f5918153670 2023-03-03T06:25:29.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.511+0000 7f590d7fa700 1 -- 172.21.15.3:0/1032223517 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908007480 con 0x7f5918153670 2023-03-03T06:25:29.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.511+0000 7f591dd97700 1 -- 172.21.15.3:0/1032223517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59180cd9a0 con 0x7f5918153670 2023-03-03T06:25:29.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.511+0000 7f591dd97700 1 -- 172.21.15.3:0/1032223517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59180cf200 con 0x7f5918153670 2023-03-03T06:25:29.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.511+0000 7f590d7fa700 1 -- 172.21.15.3:0/1032223517 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5908019030 con 0x7f5918153670 2023-03-03T06:25:29.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.515+0000 7f590d7fa700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5904064ee0 0x7f5904067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:29.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.515+0000 7f590d7fa700 1 -- 172.21.15.3:0/1032223517 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5908081520 con 0x7f5918153670 2023-03-03T06:25:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.515+0000 7f591cd95700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5904064ee0 0x7f5904067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.519+0000 7f591cd95700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5904064ee0 0x7f5904067390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59140011e0 tx=0x7f5914009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.519+0000 7f590d7fa700 1 -- 172.21.15.3:0/1032223517 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f590804f2e0 con 0x7f5918153670 2023-03-03T06:25:29.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.707+0000 7f591dd97700 1 -- 172.21.15.3:0/1032223517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59180b4020 con 0x7f5918153670 2023-03-03T06:25:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.711+0000 7f590d7fa700 1 -- 172.21.15.3:0/1032223517 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19556 (secure 0 0 0) 0x7f5908052900 con 0x7f5918153670 2023-03-03T06:25:29.715 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:29.715 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:18.863350+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:25:29.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.711+0000 7f591dd97700 1 -- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5904064ee0 msgr2=0x7f5904067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.711+0000 7f591dd97700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5904064ee0 0x7f5904067390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59140011e0 tx=0x7f5914009040 comp rx=0 tx=0).stop 2023-03-03T06:25:29.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.711+0000 7f591dd97700 1 -- 172.21.15.3:0/1032223517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918153670 msgr2=0x7f59180c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.711+0000 7f591dd97700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918153670 0x7f59180c9ce0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f590800c4d0 tx=0x7f590800c7e0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.715+0000 7f591dd97700 1 -- 172.21.15.3:0/1032223517 shutdown_connections 2023-03-03T06:25:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.715+0000 7f591dd97700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5904064ee0 0x7f5904067390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.715+0000 7f591dd97700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59181608d0 0x7f591814cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.715+0000 7f591dd97700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5918153670 0x7f59180c9ce0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.715+0000 7f591dd97700 1 --2- 172.21.15.3:0/1032223517 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f59180cfd30 0x7f591814dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.715+0000 7f591dd97700 1 -- 172.21.15.3:0/1032223517 >> 172.21.15.3:0/1032223517 conn(0x7f591806e5c0 msgr2=0x7f591815f350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.715+0000 7f591dd97700 1 -- 172.21.15.3:0/1032223517 shutdown_connections 2023-03-03T06:25:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.715+0000 7f591dd97700 1 -- 172.21.15.3:0/1032223517 wait complete. 2023-03-03T06:25:29.753 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-03T06:25:11.468534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-03T06:25:12.813283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-03T06:25:16.382800+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-03T06:25:29.754 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:25:29.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.971+0000 7f4245228700 1 Processor -- start 2023-03-03T06:25:29.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.971+0000 7f4245228700 1 -- start start 2023-03-03T06:25:29.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.975+0000 7f4245228700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4240156ba0 0x7f4240071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:29.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.975+0000 7f4245228700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42400cf3b0 con 0x7f424015d6e0 2023-03-03T06:25:29.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.975+0000 7f4245228700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4240156f70 con 0x7f42400d0b30 2023-03-03T06:25:29.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.975+0000 7f4245228700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42400cd320 con 0x7f4240156ba0 2023-03-03T06:25:29.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.975+0000 7f423f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f424015d6e0 0x7f42400cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35964/0 (socket says 172.21.15.3:35964) 2023-03-03T06:25:29.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.975+0000 7f423f7fe700 1 -- 172.21.15.3:0/2427851610 learned_addr learned my addr 172.21.15.3:0/2427851610 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:29.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.975+0000 7f423ffff700 1 --2- 172.21.15.3:0/2427851610 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4240156ba0 0x7f4240071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:29.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.975+0000 7f423effd700 1 -- 172.21.15.3:0/2427851610 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350272117 0 0) 0x7f42400cf3b0 con 0x7f424015d6e0 2023-03-03T06:25:29.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.975+0000 7f423effd700 1 -- 172.21.15.3:0/2427851610 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42280035d0 con 0x7f424015d6e0 2023-03-03T06:25:29.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f423ffff700 1 -- 172.21.15.3:0/2427851610 >> v1:172.21.15.32:6789/0 conn(0x7f42400d0b30 legacy=0x7f42400b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f423effd700 1 -- 172.21.15.3:0/2427851610 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185533950 0 0) 0x7f4240156f70 con 0x7f42400d0b30 2023-03-03T06:25:29.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f423ffff700 1 -- 172.21.15.3:0/2427851610 >> v1:172.21.15.3:6789/0 conn(0x7f424015d6e0 legacy=0x7f42400cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f423ffff700 1 -- 172.21.15.3:0/2427851610 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42400cd600 con 0x7f4240156ba0 2023-03-03T06:25:29.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f423ffff700 1 --2- 172.21.15.3:0/2427851610 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4240156ba0 0x7f4240071280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4230009dc0 tx=0x7f423000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=314cb0f27eb7f377 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:29.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f423effd700 1 -- 172.21.15.3:0/2427851610 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423000e070 con 0x7f4240156ba0 2023-03-03T06:25:29.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f423effd700 1 -- 172.21.15.3:0/2427851610 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423000bb20 con 0x7f4240156ba0 2023-03-03T06:25:29.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f423effd700 1 -- 172.21.15.3:0/2427851610 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423000bed0 con 0x7f4240156ba0 2023-03-03T06:25:29.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f4245228700 1 -- 172.21.15.3:0/2427851610 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4240156ba0 msgr2=0x7f4240071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f4245228700 1 --2- 172.21.15.3:0/2427851610 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4240156ba0 0x7f4240071280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4230009dc0 tx=0x7f423000b330 comp rx=0 tx=0).stop 2023-03-03T06:25:29.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f4245228700 1 -- 172.21.15.3:0/2427851610 shutdown_connections 2023-03-03T06:25:29.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f4245228700 1 --2- 172.21.15.3:0/2427851610 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4240156ba0 0x7f4240071280 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f4245228700 1 -- 172.21.15.3:0/2427851610 >> 172.21.15.3:0/2427851610 conn(0x7f424006e5c0 msgr2=0x7f424006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:29.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f4245228700 1 -- 172.21.15.3:0/2427851610 shutdown_connections 2023-03-03T06:25:29.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.979+0000 7f4245228700 1 -- 172.21.15.3:0/2427851610 wait complete. 2023-03-03T06:25:29.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.983+0000 7f4245228700 1 Processor -- start 2023-03-03T06:25:29.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.983+0000 7f4245228700 1 -- start start 2023-03-03T06:25:29.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f4245228700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42400d0b30 0x7f42401544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:29.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f4245228700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4240156ba0 0x7f42401510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:29.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f4245228700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f424015d6e0 0x7f4240151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:29.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f4245228700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42400b4230 con 0x7f42400d0b30 2023-03-03T06:25:29.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f4245228700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42400b31d0 con 0x7f4240156ba0 2023-03-03T06:25:29.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f4245228700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42400b3470 con 0x7f424015d6e0 2023-03-03T06:25:29.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f423ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42400d0b30 0x7f42401544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:29.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f423ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42400d0b30 0x7f42401544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44572/0 (socket says 172.21.15.3:44572) 2023-03-03T06:25:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f423ffff700 1 -- 172.21.15.3:0/1560239006 learned_addr learned my addr 172.21.15.3:0/1560239006 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f4244a27700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f424015d6e0 0x7f4240151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f423f7fe700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4240156ba0 0x7f42401510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f423ffff700 1 -- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f424015d6e0 msgr2=0x7f4240151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f423ffff700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f424015d6e0 0x7f4240151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f423ffff700 1 -- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4240156ba0 msgr2=0x7f42401510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:29.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f423ffff700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4240156ba0 0x7f42401510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f423ffff700 1 -- 172.21.15.3:0/1560239006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4230009a70 con 0x7f42400d0b30 2023-03-03T06:25:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f423ffff700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42400d0b30 0x7f42401544c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4230009f30 tx=0x7f4230004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:29.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f423cff9700 1 -- 172.21.15.3:0/1560239006 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423001b8d0 con 0x7f42400d0b30 2023-03-03T06:25:29.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f423cff9700 1 -- 172.21.15.3:0/1560239006 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4230020070 con 0x7f42400d0b30 2023-03-03T06:25:29.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f423cff9700 1 -- 172.21.15.3:0/1560239006 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423001bc50 con 0x7f42400d0b30 2023-03-03T06:25:29.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.987+0000 7f4245228700 1 -- 172.21.15.3:0/1560239006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42400717c0 con 0x7f42400d0b30 2023-03-03T06:25:29.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.991+0000 7f4245228700 1 -- 172.21.15.3:0/1560239006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42400cd8e0 con 0x7f42400d0b30 2023-03-03T06:25:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.991+0000 7f4245228700 1 -- 172.21.15.3:0/1560239006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42400cff50 con 0x7f42400d0b30 2023-03-03T06:25:29.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.995+0000 7f423cff9700 1 -- 172.21.15.3:0/1560239006 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f423002e070 con 0x7f42400d0b30 2023-03-03T06:25:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.995+0000 7f423cff9700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f422c064ee0 0x7f422c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:30.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.995+0000 7f423cff9700 1 -- 172.21.15.3:0/1560239006 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4230022070 con 0x7f42400d0b30 2023-03-03T06:25:30.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.995+0000 7f423f7fe700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f422c064ee0 0x7f422c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:30.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:29.999+0000 7f423f7fe700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f422c064ee0 0x7f422c067390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f42401f4c70 tx=0x7f4234009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.003+0000 7f423cff9700 1 -- 172.21.15.3:0/1560239006 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f42300521c0 con 0x7f42400d0b30 2023-03-03T06:25:30.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.183+0000 7f4245228700 1 -- 172.21.15.3:0/1560239006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4240073e90 con 0x7f42400d0b30 2023-03-03T06:25:30.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.183+0000 7f423cff9700 1 -- 172.21.15.3:0/1560239006 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f423002c020 con 0x7f42400d0b30 2023-03-03T06:25:30.187 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:25:30.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.187+0000 7f4245228700 1 -- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f422c064ee0 msgr2=0x7f422c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:30.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.187+0000 7f4245228700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f422c064ee0 0x7f422c067390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f42401f4c70 tx=0x7f4234009040 comp rx=0 tx=0).stop 2023-03-03T06:25:30.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.187+0000 7f4245228700 1 -- 172.21.15.3:0/1560239006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42400d0b30 msgr2=0x7f42401544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:30.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.187+0000 7f4245228700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42400d0b30 0x7f42401544c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4230009f30 tx=0x7f4230004af0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.187+0000 7f4245228700 1 -- 172.21.15.3:0/1560239006 shutdown_connections 2023-03-03T06:25:30.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.187+0000 7f4245228700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f422c064ee0 0x7f422c067390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.187+0000 7f4245228700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f424015d6e0 0x7f4240151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.187+0000 7f4245228700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4240156ba0 0x7f42401510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.187+0000 7f4245228700 1 --2- 172.21.15.3:0/1560239006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42400d0b30 0x7f42401544c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.187+0000 7f4245228700 1 -- 172.21.15.3:0/1560239006 >> 172.21.15.3:0/1560239006 conn(0x7f424006e5c0 msgr2=0x7f4240160860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:30.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.187+0000 7f4245228700 1 -- 172.21.15.3:0/1560239006 shutdown_connections 2023-03-03T06:25:30.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.187+0000 7f4245228700 1 -- 172.21.15.3:0/1560239006 wait complete. 2023-03-03T06:25:30.226 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:25:30.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.439+0000 7f48da6cb700 1 Processor -- start 2023-03-03T06:25:30.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.439+0000 7f48da6cb700 1 -- start start 2023-03-03T06:25:30.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.443+0000 7f48da6cb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48d415d6e0 0x7f48d40cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:30.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.443+0000 7f48da6cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48d40cfd80 0x7f48d40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:30.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.443+0000 7f48da6cb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d40cf3b0 con 0x7f48d4156ba0 2023-03-03T06:25:30.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.443+0000 7f48da6cb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d4156f70 con 0x7f48d415d6e0 2023-03-03T06:25:30.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.443+0000 7f48da6cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40cd320 con 0x7f48d40cfd80 2023-03-03T06:25:30.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.443+0000 7f48d9eca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48d40cfd80 0x7f48d40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:30.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.443+0000 7f48d8ec8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48d415d6e0 0x7f48d40cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:30.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.447+0000 7f48d9eca700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48d40cfd80 0x7f48d40d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:55550/0 (socket says 172.21.15.3:55550) 2023-03-03T06:25:30.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.447+0000 7f48d9eca700 1 -- 172.21.15.3:0/1389398445 learned_addr learned my addr 172.21.15.3:0/1389398445 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.447+0000 7f48d9eca700 1 -- 172.21.15.3:0/1389398445 >> v2:172.21.15.32:3300/0 conn(0x7f48d415d6e0 msgr2=0x7f48d40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.447+0000 7f48d9eca700 1 --2- 172.21.15.3:0/1389398445 >> v2:172.21.15.32:3300/0 conn(0x7f48d415d6e0 0x7f48d40cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.447+0000 7f48d9eca700 1 -- 172.21.15.3:0/1389398445 >> v1:172.21.15.32:6789/0 conn(0x7f48d4156ba0 legacy=0x7f48d4071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.447+0000 7f48d9eca700 1 -- 172.21.15.3:0/1389398445 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d40cd600 con 0x7f48d40cfd80 2023-03-03T06:25:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.447+0000 7f48d9eca700 1 --2- 172.21.15.3:0/1389398445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48d40cfd80 0x7f48d40d0b30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009e50 tx=0x7f48cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b60b445c6350081 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.447+0000 7f48cbfff700 1 -- 172.21.15.3:0/1389398445 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00e070 con 0x7f48d40cfd80 2023-03-03T06:25:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.447+0000 7f48cbfff700 1 -- 172.21.15.3:0/1389398445 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc00b910 con 0x7f48d40cfd80 2023-03-03T06:25:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.447+0000 7f48cbfff700 1 -- 172.21.15.3:0/1389398445 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00bcc0 con 0x7f48d40cfd80 2023-03-03T06:25:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.447+0000 7f48da6cb700 1 -- 172.21.15.3:0/1389398445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48d40cfd80 msgr2=0x7f48d40d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:30.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.447+0000 7f48da6cb700 1 --2- 172.21.15.3:0/1389398445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48d40cfd80 0x7f48d40d0b30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009e50 tx=0x7f48cc00b330 comp rx=0 tx=0).stop 2023-03-03T06:25:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.451+0000 7f48da6cb700 1 -- 172.21.15.3:0/1389398445 shutdown_connections 2023-03-03T06:25:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.451+0000 7f48da6cb700 1 --2- 172.21.15.3:0/1389398445 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48d40cfd80 0x7f48d40d0b30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.451+0000 7f48da6cb700 1 --2- 172.21.15.3:0/1389398445 >> v2:172.21.15.32:3300/0 conn(0x7f48d415d6e0 0x7f48d40cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.451+0000 7f48da6cb700 1 -- 172.21.15.3:0/1389398445 >> 172.21.15.3:0/1389398445 conn(0x7f48d406e5c0 msgr2=0x7f48d406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.451+0000 7f48da6cb700 1 -- 172.21.15.3:0/1389398445 shutdown_connections 2023-03-03T06:25:30.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.451+0000 7f48da6cb700 1 -- 172.21.15.3:0/1389398445 wait complete. 2023-03-03T06:25:30.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.451+0000 7f48da6cb700 1 Processor -- start 2023-03-03T06:25:30.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48da6cb700 1 -- start start 2023-03-03T06:25:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48da6cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48d40cfd80 0x7f48d40b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:30.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48da6cb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d4156ba0 0x7f48d40cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48da6cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48d415d6e0 0x7f48d4150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48da6cb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d4150650 con 0x7f48d4156ba0 2023-03-03T06:25:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48da6cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40cb2f0 con 0x7f48d40cfd80 2023-03-03T06:25:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48da6cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40cb5d0 con 0x7f48d415d6e0 2023-03-03T06:25:30.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48d8ec8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d4156ba0 0x7f48d40cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:30.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48d8ec8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d4156ba0 0x7f48d40cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52762/0 (socket says 172.21.15.3:52762) 2023-03-03T06:25:30.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48d8ec8700 1 -- 172.21.15.3:0/4157308944 learned_addr learned my addr 172.21.15.3:0/4157308944 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:30.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48d9eca700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48d415d6e0 0x7f48d4150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:30.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48d96c9700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48d40cfd80 0x7f48d40b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:30.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48d8ec8700 1 -- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48d415d6e0 msgr2=0x7f48d4150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:30.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48d8ec8700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48d415d6e0 0x7f48d4150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48d8ec8700 1 -- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48d40cfd80 msgr2=0x7f48d40b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:30.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48d8ec8700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48d40cfd80 0x7f48d40b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.455+0000 7f48d8ec8700 1 -- 172.21.15.3:0/4157308944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc009b30 con 0x7f48d4156ba0 2023-03-03T06:25:30.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.459+0000 7f48d8ec8700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d4156ba0 0x7f48d40cc390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f48c400e9d0 tx=0x7f48c400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:30.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.459+0000 7f48c9ffb700 1 -- 172.21.15.3:0/4157308944 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4009720 con 0x7f48d4156ba0 2023-03-03T06:25:30.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.459+0000 7f48c9ffb700 1 -- 172.21.15.3:0/4157308944 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c400ee70 con 0x7f48d4156ba0 2023-03-03T06:25:30.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.459+0000 7f48c9ffb700 1 -- 172.21.15.3:0/4157308944 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c400ca50 con 0x7f48d4156ba0 2023-03-03T06:25:30.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.459+0000 7f48da6cb700 1 -- 172.21.15.3:0/4157308944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d40cef40 con 0x7f48d4156ba0 2023-03-03T06:25:30.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.459+0000 7f48da6cb700 1 -- 172.21.15.3:0/4157308944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d40cd8e0 con 0x7f48d4156ba0 2023-03-03T06:25:30.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.459+0000 7f48da6cb700 1 -- 172.21.15.3:0/4157308944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d40cf0d0 con 0x7f48d4156ba0 2023-03-03T06:25:30.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.463+0000 7f48c9ffb700 1 -- 172.21.15.3:0/4157308944 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f48c4019030 con 0x7f48d4156ba0 2023-03-03T06:25:30.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.467+0000 7f48c9ffb700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f48b8064ee0 0x7f48b8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:30.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.467+0000 7f48c9ffb700 1 -- 172.21.15.3:0/4157308944 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f48c4082150 con 0x7f48d4156ba0 2023-03-03T06:25:30.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.467+0000 7f48d96c9700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f48b8064ee0 0x7f48b8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:30.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.467+0000 7f48d96c9700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f48b8064ee0 0x7f48b8067390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48d0000f60 tx=0x7f48d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:30.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.471+0000 7f48c9ffb700 1 -- 172.21.15.3:0/4157308944 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f48c404feb0 con 0x7f48d4156ba0 2023-03-03T06:25:30.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.647+0000 7f48da6cb700 1 -- 172.21.15.3:0/4157308944 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f48d4073e90 con 0x7f48d4156ba0 2023-03-03T06:25:30.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.647+0000 7f48c9ffb700 1 -- 172.21.15.3:0/4157308944 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f48c40534d0 con 0x7f48d4156ba0 2023-03-03T06:25:30.653 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:25:30.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.655+0000 7f48da6cb700 1 -- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f48b8064ee0 msgr2=0x7f48b8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:30.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.655+0000 7f48da6cb700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f48b8064ee0 0x7f48b8067390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48d0000f60 tx=0x7f48d0009040 comp rx=0 tx=0).stop 2023-03-03T06:25:30.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.655+0000 7f48da6cb700 1 -- 172.21.15.3:0/4157308944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d4156ba0 msgr2=0x7f48d40cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:30.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.655+0000 7f48da6cb700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d4156ba0 0x7f48d40cc390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f48c400e9d0 tx=0x7f48c400ece0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.655+0000 7f48da6cb700 1 -- 172.21.15.3:0/4157308944 shutdown_connections 2023-03-03T06:25:30.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.655+0000 7f48da6cb700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f48b8064ee0 0x7f48b8067390 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.655+0000 7f48da6cb700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48d415d6e0 0x7f48d4150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.655+0000 7f48da6cb700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48d4156ba0 0x7f48d40cc390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.655+0000 7f48da6cb700 1 --2- 172.21.15.3:0/4157308944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48d40cfd80 0x7f48d40b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.655+0000 7f48da6cb700 1 -- 172.21.15.3:0/4157308944 >> 172.21.15.3:0/4157308944 conn(0x7f48d406e5c0 msgr2=0x7f48d40d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:30.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.655+0000 7f48da6cb700 1 -- 172.21.15.3:0/4157308944 shutdown_connections 2023-03-03T06:25:30.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.655+0000 7f48da6cb700 1 -- 172.21.15.3:0/4157308944 wait complete. 2023-03-03T06:25:30.691 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-03T06:25:30.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.899+0000 7efee88ff700 1 Processor -- start 2023-03-03T06:25:30.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.899+0000 7efee88ff700 1 -- start start 2023-03-03T06:25:30.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.903+0000 7efee88ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee40b4d90 0x7efee40b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.903+0000 7efee88ff700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee40cd2f0 con 0x7efee40cd7d0 2023-03-03T06:25:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.903+0000 7efee88ff700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee40cf240 con 0x7efee415faf0 2023-03-03T06:25:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.903+0000 7efee88ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee41609d0 con 0x7efee40b4d90 2023-03-03T06:25:30.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.903+0000 7efee3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efee40cd7d0 0x7efee40d07d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41648/0 (socket says 172.21.15.3:41648) 2023-03-03T06:25:30.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.903+0000 7efee3fff700 1 -- 172.21.15.3:0/1117242998 learned_addr learned my addr 172.21.15.3:0/1117242998 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:30.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.903+0000 7efee2ffd700 1 --2- 172.21.15.3:0/1117242998 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee40b4d90 0x7efee40b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:30.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.903+0000 7efee27fc700 1 -- 172.21.15.3:0/1117242998 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812247748 0 0) 0x7efee40cd2f0 con 0x7efee40cd7d0 2023-03-03T06:25:30.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee27fc700 1 -- 172.21.15.3:0/1117242998 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efecc0035d0 con 0x7efee40cd7d0 2023-03-03T06:25:30.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee27fc700 1 -- 172.21.15.3:0/1117242998 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896096531 0 0) 0x7efee40cf240 con 0x7efee415faf0 2023-03-03T06:25:30.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee27fc700 1 -- 172.21.15.3:0/1117242998 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee40cd2f0 con 0x7efee415faf0 2023-03-03T06:25:30.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee2ffd700 1 -- 172.21.15.3:0/1117242998 >> v1:172.21.15.32:6789/0 conn(0x7efee415faf0 legacy=0x7efee40b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:30.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee2ffd700 1 -- 172.21.15.3:0/1117242998 >> v1:172.21.15.3:6789/0 conn(0x7efee40cd7d0 legacy=0x7efee40d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:30.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee2ffd700 1 -- 172.21.15.3:0/1117242998 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee4160bc0 con 0x7efee40b4d90 2023-03-03T06:25:30.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee2ffd700 1 --2- 172.21.15.3:0/1117242998 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee40b4d90 0x7efee40b1ac0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efed8009880 tx=0x7efed8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=641dcce0307859e server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:30.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee27fc700 1 -- 172.21.15.3:0/1117242998 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8011830 con 0x7efee40b4d90 2023-03-03T06:25:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee27fc700 1 -- 172.21.15.3:0/1117242998 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed8011990 con 0x7efee40b4d90 2023-03-03T06:25:30.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee27fc700 1 -- 172.21.15.3:0/1117242998 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8011d20 con 0x7efee40b4d90 2023-03-03T06:25:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee88ff700 1 -- 172.21.15.3:0/1117242998 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee40b4d90 msgr2=0x7efee40b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee88ff700 1 --2- 172.21.15.3:0/1117242998 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee40b4d90 0x7efee40b1ac0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efed8009880 tx=0x7efed8009b90 comp rx=0 tx=0).stop 2023-03-03T06:25:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee88ff700 1 -- 172.21.15.3:0/1117242998 shutdown_connections 2023-03-03T06:25:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee88ff700 1 --2- 172.21.15.3:0/1117242998 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee40b4d90 0x7efee40b1ac0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee88ff700 1 -- 172.21.15.3:0/1117242998 >> 172.21.15.3:0/1117242998 conn(0x7efee40cf680 msgr2=0x7efee40cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee88ff700 1 -- 172.21.15.3:0/1117242998 shutdown_connections 2023-03-03T06:25:30.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.907+0000 7efee88ff700 1 -- 172.21.15.3:0/1117242998 wait complete. 2023-03-03T06:25:30.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.911+0000 7efee88ff700 1 Processor -- start 2023-03-03T06:25:30.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.911+0000 7efee88ff700 1 -- start start 2023-03-03T06:25:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee88ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee40b4d90 0x7efee40c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:30.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee88ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee40cd7d0 0x7efee414c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee88ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee415faf0 0x7efee41f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee88ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee414ca40 con 0x7efee40b4d90 2023-03-03T06:25:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee88ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee41f1c20 con 0x7efee40cd7d0 2023-03-03T06:25:30.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee88ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee40ca3b0 con 0x7efee415faf0 2023-03-03T06:25:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee40b4d90 0x7efee40c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee40b4d90 0x7efee40c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52774/0 (socket says 172.21.15.3:52774) 2023-03-03T06:25:30.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee37fe700 1 -- 172.21.15.3:0/4166768828 learned_addr learned my addr 172.21.15.3:0/4166768828 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:30.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee2ffd700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee40cd7d0 0x7efee414c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:30.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee3fff700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee415faf0 0x7efee41f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:30.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee37fe700 1 -- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee415faf0 msgr2=0x7efee41f16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:30.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee37fe700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee415faf0 0x7efee41f16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee37fe700 1 -- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee40cd7d0 msgr2=0x7efee414c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:30.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee37fe700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee40cd7d0 0x7efee414c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:30.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee37fe700 1 -- 172.21.15.3:0/4166768828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed80094b0 con 0x7efee40b4d90 2023-03-03T06:25:30.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee3fff700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee415faf0 0x7efee41f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:25:30.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee37fe700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee40b4d90 0x7efee40c9cd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efed400c490 tx=0x7efed400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:30.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efecbfff700 1 -- 172.21.15.3:0/4166768828 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed40052d0 con 0x7efee40b4d90 2023-03-03T06:25:30.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efecbfff700 1 -- 172.21.15.3:0/4166768828 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed4005450 con 0x7efee40b4d90 2023-03-03T06:25:30.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efecbfff700 1 -- 172.21.15.3:0/4166768828 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed40038e0 con 0x7efee40b4d90 2023-03-03T06:25:30.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee88ff700 1 -- 172.21.15.3:0/4166768828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee40ceef0 con 0x7efee40b4d90 2023-03-03T06:25:30.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.915+0000 7efee88ff700 1 -- 172.21.15.3:0/4166768828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee4160ea0 con 0x7efee40b4d90 2023-03-03T06:25:30.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.919+0000 7efee88ff700 1 -- 172.21.15.3:0/4166768828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee40b2070 con 0x7efee40b4d90 2023-03-03T06:25:30.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.923+0000 7efecbfff700 1 -- 172.21.15.3:0/4166768828 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7efed4019030 con 0x7efee40b4d90 2023-03-03T06:25:30.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.923+0000 7efecbfff700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efed0064ee0 0x7efed0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:30.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.923+0000 7efecbfff700 1 -- 172.21.15.3:0/4166768828 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efed4081540 con 0x7efee40b4d90 2023-03-03T06:25:30.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.923+0000 7efee2ffd700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efed0064ee0 0x7efed0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:30.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.923+0000 7efee2ffd700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efed0064ee0 0x7efed0067390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efee40b5180 tx=0x7efed801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:30.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:30.927+0000 7efecbfff700 1 -- 172.21.15.3:0/4166768828 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7efed404f3f0 con 0x7efee40b4d90 2023-03-03T06:25:31.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.103+0000 7efee88ff700 1 -- 172.21.15.3:0/4166768828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efee4073e90 con 0x7efee40b4d90 2023-03-03T06:25:31.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.103+0000 7efecbfff700 1 -- 172.21.15.3:0/4166768828 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7efed4052a10 con 0x7efee40b4d90 2023-03-03T06:25:31.108 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:25:31.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.107+0000 7efee88ff700 1 -- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efed0064ee0 msgr2=0x7efed0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:31.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.107+0000 7efee88ff700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efed0064ee0 0x7efed0067390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efee40b5180 tx=0x7efed801c040 comp rx=0 tx=0).stop 2023-03-03T06:25:31.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.107+0000 7efee88ff700 1 -- 172.21.15.3:0/4166768828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee40b4d90 msgr2=0x7efee40c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:31.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.107+0000 7efee88ff700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee40b4d90 0x7efee40c9cd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efed400c490 tx=0x7efed400c850 comp rx=0 tx=0).stop 2023-03-03T06:25:31.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.107+0000 7efee88ff700 1 -- 172.21.15.3:0/4166768828 shutdown_connections 2023-03-03T06:25:31.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.107+0000 7efee88ff700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efed0064ee0 0x7efed0067390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.107+0000 7efee88ff700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efee415faf0 0x7efee41f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.107+0000 7efee88ff700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee40cd7d0 0x7efee414c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.107+0000 7efee88ff700 1 --2- 172.21.15.3:0/4166768828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efee40b4d90 0x7efee40c9cd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.107+0000 7efee88ff700 1 -- 172.21.15.3:0/4166768828 >> 172.21.15.3:0/4166768828 conn(0x7efee40cf680 msgr2=0x7efee40b7440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:31.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.107+0000 7efee88ff700 1 -- 172.21.15.3:0/4166768828 shutdown_connections 2023-03-03T06:25:31.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.111+0000 7efee88ff700 1 -- 172.21.15.3:0/4166768828 wait complete. 2023-03-03T06:25:31.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-03T06:25:31.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.367+0000 7ff0e359e700 1 Processor -- start 2023-03-03T06:25:31.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.367+0000 7ff0e359e700 1 -- start start 2023-03-03T06:25:31.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0e40cb9a0 0x7ff0e40cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:31.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e359e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e415c230 con 0x7ff0e406a780 2023-03-03T06:25:31.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e359e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e40cd130 con 0x7ff0e40710c0 2023-03-03T06:25:31.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e40caac0 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e259c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff0e406a780 0x7ff0e40709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41658/0 (socket says 172.21.15.3:41658) 2023-03-03T06:25:31.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e259c700 1 -- 172.21.15.3:0/4258224710 learned_addr learned my addr 172.21.15.3:0/4258224710 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:31.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e2d9d700 1 --2- 172.21.15.3:0/4258224710 >> v2:172.21.15.32:3300/0 conn(0x7ff0e40cb9a0 0x7ff0e40cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:31.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e2d9d700 1 -- 172.21.15.3:0/4258224710 >> v1:172.21.15.32:6789/0 conn(0x7ff0e40710c0 legacy=0x7ff0e40ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:31.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e2d9d700 1 -- 172.21.15.3:0/4258224710 >> v1:172.21.15.3:6789/0 conn(0x7ff0e406a780 legacy=0x7ff0e40709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:31.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e2d9d700 1 -- 172.21.15.3:0/4258224710 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e40cbf60 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e1d9b700 1 -- 172.21.15.3:0/4258224710 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406629745 0 0) 0x7ff0e415c230 con 0x7ff0e406a780 2023-03-03T06:25:31.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e2d9d700 1 --2- 172.21.15.3:0/4258224710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e40cb9a0 0x7ff0e40cace0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009d90 tx=0x7ff0dc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c64bccf61be79e9 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:31.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e1d9b700 1 -- 172.21.15.3:0/4258224710 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc00e070 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e1d9b700 1 -- 172.21.15.3:0/4258224710 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc00bbd0 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.371+0000 7ff0e1d9b700 1 -- 172.21.15.3:0/4258224710 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc012430 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.375+0000 7ff0e359e700 1 -- 172.21.15.3:0/4258224710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e40cb9a0 msgr2=0x7ff0e40cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:31.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.375+0000 7ff0e359e700 1 --2- 172.21.15.3:0/4258224710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e40cb9a0 0x7ff0e40cace0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009d90 tx=0x7ff0dc00b330 comp rx=0 tx=0).stop 2023-03-03T06:25:31.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.375+0000 7ff0e359e700 1 -- 172.21.15.3:0/4258224710 shutdown_connections 2023-03-03T06:25:31.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.375+0000 7ff0e359e700 1 --2- 172.21.15.3:0/4258224710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e40cb9a0 0x7ff0e40cace0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.375+0000 7ff0e359e700 1 -- 172.21.15.3:0/4258224710 >> 172.21.15.3:0/4258224710 conn(0x7ff0e40607c0 msgr2=0x7ff0e4060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:31.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.375+0000 7ff0e359e700 1 -- 172.21.15.3:0/4258224710 shutdown_connections 2023-03-03T06:25:31.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.375+0000 7ff0e359e700 1 -- 172.21.15.3:0/4258224710 wait complete. 2023-03-03T06:25:31.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e359e700 1 Processor -- start 2023-03-03T06:25:31.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e359e700 1 -- start start 2023-03-03T06:25:31.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e359e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0e406a780 0x7ff0e4145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:31.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e40710c0 0x7ff0e40aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:31.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0e40cb9a0 0x7ff0e40aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:31.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e359e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4146f20 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4149070 con 0x7ff0e40710c0 2023-03-03T06:25:31.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e359e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40afb50 con 0x7ff0e406a780 2023-03-03T06:25:31.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0e40cb9a0 0x7ff0e40aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:31.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0e40cb9a0 0x7ff0e40aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52782/0 (socket says 172.21.15.3:52782) 2023-03-03T06:25:31.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e2d9d700 1 -- 172.21.15.3:0/1479969869 learned_addr learned my addr 172.21.15.3:0/1479969869 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:31.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0d9d9b700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e40710c0 0x7ff0e40aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:31.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e259c700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0e406a780 0x7ff0e4145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:31.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e2d9d700 1 -- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0e406a780 msgr2=0x7ff0e4145990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:31.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e2d9d700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0e406a780 0x7ff0e4145990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e2d9d700 1 -- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e40710c0 msgr2=0x7ff0e40aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:31.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e2d9d700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e40710c0 0x7ff0e40aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.379+0000 7ff0e2d9d700 1 -- 172.21.15.3:0/1479969869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc009a70 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.383+0000 7ff0e2d9d700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0e40cb9a0 0x7ff0e40aefe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc014040 tx=0x7ff0dc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:31.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.383+0000 7ff0db7fe700 1 -- 172.21.15.3:0/1479969869 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc01b810 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.383+0000 7ff0db7fe700 1 -- 172.21.15.3:0/1479969869 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc020070 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.383+0000 7ff0db7fe700 1 -- 172.21.15.3:0/1479969869 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc01bbb0 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.383+0000 7ff0e359e700 1 -- 172.21.15.3:0/1479969869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e415c7d0 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.383+0000 7ff0e359e700 1 -- 172.21.15.3:0/1479969869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e415c4f0 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.383+0000 7ff0e359e700 1 -- 172.21.15.3:0/1479969869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e415cab0 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.387+0000 7ff0db7fe700 1 -- 172.21.15.3:0/1479969869 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff0dc02e070 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.387+0000 7ff0db7fe700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff0c4064ee0 0x7ff0c4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:31.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.387+0000 7ff0db7fe700 1 -- 172.21.15.3:0/1479969869 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff0dc022070 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.387+0000 7ff0e259c700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff0c4064ee0 0x7ff0c4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:31.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.391+0000 7ff0e259c700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff0c4064ee0 0x7ff0c4067390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4007b30 tx=0x7ff0d4007a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:31.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.395+0000 7ff0db7fe700 1 -- 172.21.15.3:0/1479969869 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff0dc052060 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.571+0000 7ff0e359e700 1 -- 172.21.15.3:0/1479969869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0e4073e90 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.575+0000 7ff0db7fe700 1 -- 172.21.15.3:0/1479969869 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0dc02c020 con 0x7ff0e40cb9a0 2023-03-03T06:25:31.578 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-03-03T06:25:31.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.575+0000 7ff0e359e700 1 -- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff0c4064ee0 msgr2=0x7ff0c4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:31.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.575+0000 7ff0e359e700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff0c4064ee0 0x7ff0c4067390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4007b30 tx=0x7ff0d4007a80 comp rx=0 tx=0).stop 2023-03-03T06:25:31.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.575+0000 7ff0e359e700 1 -- 172.21.15.3:0/1479969869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0e40cb9a0 msgr2=0x7ff0e40aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:31.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.575+0000 7ff0e359e700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0e40cb9a0 0x7ff0e40aefe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc014040 tx=0x7ff0dc004af0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.575+0000 7ff0e359e700 1 -- 172.21.15.3:0/1479969869 shutdown_connections 2023-03-03T06:25:31.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.575+0000 7ff0e359e700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff0c4064ee0 0x7ff0c4067390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.575+0000 7ff0e359e700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0e40cb9a0 0x7ff0e40aefe0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.575+0000 7ff0e359e700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0e40710c0 0x7ff0e40aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.575+0000 7ff0e359e700 1 --2- 172.21.15.3:0/1479969869 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0e406a780 0x7ff0e4145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.575+0000 7ff0e359e700 1 -- 172.21.15.3:0/1479969869 >> 172.21.15.3:0/1479969869 conn(0x7ff0e40607c0 msgr2=0x7ff0e4153c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:31.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.579+0000 7ff0e359e700 1 -- 172.21.15.3:0/1479969869 shutdown_connections 2023-03-03T06:25:31.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.579+0000 7ff0e359e700 1 -- 172.21.15.3:0/1479969869 wait complete. 2023-03-03T06:25:31.617 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-03T06:25:31.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.827+0000 7ff695839700 1 Processor -- start 2023-03-03T06:25:31.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.827+0000 7ff695839700 1 -- start start 2023-03-03T06:25:31.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff695839700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff6900b0ed0 0x7ff6900ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff695839700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff690160b60 0x7ff690164490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff695839700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6900ccc20 con 0x7ff690160790 2023-03-03T06:25:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff695839700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6900cc380 con 0x7ff6900b0ed0 2023-03-03T06:25:31.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff695839700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6900cc660 con 0x7ff690160b60 2023-03-03T06:25:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff68ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff6900b0ed0 0x7ff6900ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff68ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff6900b0ed0 0x7ff6900ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52792/0 (socket says 172.21.15.3:52792) 2023-03-03T06:25:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff68ffff700 1 -- 172.21.15.3:0/102204683 learned_addr learned my addr 172.21.15.3:0/102204683 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:31.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff695038700 1 --2- 172.21.15.3:0/102204683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff690160b60 0x7ff690164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff68ffff700 1 -- 172.21.15.3:0/102204683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff690160b60 msgr2=0x7ff690164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff68ffff700 1 --2- 172.21.15.3:0/102204683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff690160b60 0x7ff690164490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff68ffff700 1 -- 172.21.15.3:0/102204683 >> v1:172.21.15.3:6789/0 conn(0x7ff690160790 legacy=0x7ff6900b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff68ffff700 1 -- 172.21.15.3:0/102204683 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6900cc940 con 0x7ff6900b0ed0 2023-03-03T06:25:31.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.831+0000 7ff68ffff700 1 --2- 172.21.15.3:0/102204683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6900b0ed0 0x7ff6900ce060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff67c009910 tx=0x7ff67c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a1314cd2d4b933 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.835+0000 7ff68f7fe700 1 -- 172.21.15.3:0/102204683 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c00ba60 con 0x7ff6900b0ed0 2023-03-03T06:25:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.835+0000 7ff68f7fe700 1 -- 172.21.15.3:0/102204683 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67c00bbc0 con 0x7ff6900b0ed0 2023-03-03T06:25:31.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.835+0000 7ff68f7fe700 1 -- 172.21.15.3:0/102204683 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c009e60 con 0x7ff6900b0ed0 2023-03-03T06:25:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.835+0000 7ff695839700 1 -- 172.21.15.3:0/102204683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6900b0ed0 msgr2=0x7ff6900ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.835+0000 7ff695839700 1 --2- 172.21.15.3:0/102204683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6900b0ed0 0x7ff6900ce060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff67c009910 tx=0x7ff67c009c20 comp rx=0 tx=0).stop 2023-03-03T06:25:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.835+0000 7ff695839700 1 -- 172.21.15.3:0/102204683 shutdown_connections 2023-03-03T06:25:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.835+0000 7ff695839700 1 --2- 172.21.15.3:0/102204683 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff690160b60 0x7ff690164490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.835+0000 7ff695839700 1 --2- 172.21.15.3:0/102204683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6900b0ed0 0x7ff6900ce060 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.835+0000 7ff695839700 1 -- 172.21.15.3:0/102204683 >> 172.21.15.3:0/102204683 conn(0x7ff6900607c0 msgr2=0x7ff690060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:31.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.835+0000 7ff695839700 1 -- 172.21.15.3:0/102204683 shutdown_connections 2023-03-03T06:25:31.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.835+0000 7ff695839700 1 -- 172.21.15.3:0/102204683 wait complete. 2023-03-03T06:25:31.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.839+0000 7ff695839700 1 Processor -- start 2023-03-03T06:25:31.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.839+0000 7ff695839700 1 -- start start 2023-03-03T06:25:31.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.839+0000 7ff695839700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6900b0ed0 0x7ff690156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:31.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.839+0000 7ff695839700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff690160790 0x7ff6901622a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:31.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.839+0000 7ff695839700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff690160b60 0x7ff690159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:31.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.839+0000 7ff695839700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff690157310 con 0x7ff690160b60 2023-03-03T06:25:31.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.839+0000 7ff695839700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff690158f70 con 0x7ff690160790 2023-03-03T06:25:31.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.839+0000 7ff695839700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff690158080 con 0x7ff6900b0ed0 2023-03-03T06:25:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.839+0000 7ff695038700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff690160b60 0x7ff690159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff695038700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff690160b60 0x7ff690159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52804/0 (socket says 172.21.15.3:52804) 2023-03-03T06:25:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff695038700 1 -- 172.21.15.3:0/2177966714 learned_addr learned my addr 172.21.15.3:0/2177966714 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff694837700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6900b0ed0 0x7ff690156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:31.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff68ffff700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff690160790 0x7ff6901622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:31.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff695038700 1 -- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6900b0ed0 msgr2=0x7ff690156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:31.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff695038700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6900b0ed0 0x7ff690156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff695038700 1 -- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff690160790 msgr2=0x7ff6901622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff695038700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff690160790 0x7ff6901622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff695038700 1 -- 172.21.15.3:0/2177966714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c009570 con 0x7ff690160b60 2023-03-03T06:25:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff68ffff700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff690160790 0x7ff6901622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff695038700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff690160b60 0x7ff690159400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff688007d70 tx=0x7ff68800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:31.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff68d7fa700 1 -- 172.21.15.3:0/2177966714 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68800ede0 con 0x7ff690160b60 2023-03-03T06:25:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff68d7fa700 1 -- 172.21.15.3:0/2177966714 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff688014420 con 0x7ff690160b60 2023-03-03T06:25:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff695839700 1 -- 172.21.15.3:0/2177966714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6900cb980 con 0x7ff690160b60 2023-03-03T06:25:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff68d7fa700 1 -- 172.21.15.3:0/2177966714 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6880147d0 con 0x7ff690160b60 2023-03-03T06:25:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff695839700 1 -- 172.21.15.3:0/2177966714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6900cb6a0 con 0x7ff690160b60 2023-03-03T06:25:31.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.843+0000 7ff695839700 1 -- 172.21.15.3:0/2177966714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6900cbea0 con 0x7ff690160b60 2023-03-03T06:25:31.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.847+0000 7ff68d7fa700 1 -- 172.21.15.3:0/2177966714 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff688012050 con 0x7ff690160b60 2023-03-03T06:25:31.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.847+0000 7ff68d7fa700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff674064ee0 0x7ff674067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:31.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.847+0000 7ff68d7fa700 1 -- 172.21.15.3:0/2177966714 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff688082800 con 0x7ff690160b60 2023-03-03T06:25:31.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.851+0000 7ff694837700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff674064ee0 0x7ff674067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:31.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.851+0000 7ff694837700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff674064ee0 0x7ff674067390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff684000f60 tx=0x7ff684009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:31.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:31.855+0000 7ff68d7fa700 1 -- 172.21.15.3:0/2177966714 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff688050540 con 0x7ff690160b60 2023-03-03T06:25:32.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.031+0000 7ff695839700 1 -- 172.21.15.3:0/2177966714 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff690073e90 con 0x7ff690160b60 2023-03-03T06:25:32.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.035+0000 7ff68d7fa700 1 -- 172.21.15.3:0/2177966714 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff688053b60 con 0x7ff690160b60 2023-03-03T06:25:32.038 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:25:32.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.035+0000 7ff695839700 1 -- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff674064ee0 msgr2=0x7ff674067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:32.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.035+0000 7ff695839700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff674064ee0 0x7ff674067390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff684000f60 tx=0x7ff684009040 comp rx=0 tx=0).stop 2023-03-03T06:25:32.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.035+0000 7ff695839700 1 -- 172.21.15.3:0/2177966714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff690160b60 msgr2=0x7ff690159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:32.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.035+0000 7ff695839700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff690160b60 0x7ff690159400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff688007d70 tx=0x7ff68800e3c0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.039+0000 7ff695839700 1 -- 172.21.15.3:0/2177966714 shutdown_connections 2023-03-03T06:25:32.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.039+0000 7ff695839700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff674064ee0 0x7ff674067390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.039+0000 7ff695839700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff690160b60 0x7ff690159400 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.039+0000 7ff695839700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff690160790 0x7ff6901622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.039+0000 7ff695839700 1 --2- 172.21.15.3:0/2177966714 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff6900b0ed0 0x7ff690156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.039+0000 7ff695839700 1 -- 172.21.15.3:0/2177966714 >> 172.21.15.3:0/2177966714 conn(0x7ff6900607c0 msgr2=0x7ff69015d440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:32.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.039+0000 7ff695839700 1 -- 172.21.15.3:0/2177966714 shutdown_connections 2023-03-03T06:25:32.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.039+0000 7ff695839700 1 -- 172.21.15.3:0/2177966714 wait complete. 2023-03-03T06:25:32.076 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-03T06:25:32.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.279+0000 7fc12a526700 1 Processor -- start 2023-03-03T06:25:32.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc12a526700 1 -- start start 2023-03-03T06:25:32.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc12a526700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1240cb890 0x7fc124163650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:32.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc12a526700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1240cceb0 con 0x7fc1240adcd0 2023-03-03T06:25:32.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc12a526700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1240cd340 con 0x7fc124158ab0 2023-03-03T06:25:32.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc12a526700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1240ce3f0 con 0x7fc1240cb890 2023-03-03T06:25:32.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc128d23700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc1240adcd0 0x7fc1240cb180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41680/0 (socket says 172.21.15.3:41680) 2023-03-03T06:25:32.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc128d23700 1 -- 172.21.15.3:0/3197451536 learned_addr learned my addr 172.21.15.3:0/3197451536 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:32.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc129d25700 1 --2- 172.21.15.3:0/3197451536 >> v2:172.21.15.32:3300/0 conn(0x7fc1240cb890 0x7fc124163650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777610412 0 0) 0x7fc1240cceb0 con 0x7fc1240adcd0 2023-03-03T06:25:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1100057e0 con 0x7fc1240adcd0 2023-03-03T06:25:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363515378 0 0) 0x7fc1240cd340 con 0x7fc124158ab0 2023-03-03T06:25:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1240cceb0 con 0x7fc124158ab0 2023-03-03T06:25:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2656030844 0 0) 0x7fc1100057e0 con 0x7fc1240adcd0 2023-03-03T06:25:32.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 >> v2:172.21.15.32:3300/0 conn(0x7fc1240cb890 msgr2=0x7fc124163650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:32.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc11bfff700 1 --2- 172.21.15.3:0/3197451536 >> v2:172.21.15.32:3300/0 conn(0x7fc1240cb890 0x7fc124163650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 >> v1:172.21.15.32:6789/0 conn(0x7fc124158ab0 legacy=0x7fc1240cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:32.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1240cc580 con 0x7fc1240adcd0 2023-03-03T06:25:32.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7fc114001670 con 0x7fc1240adcd0 2023-03-03T06:25:32.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.283+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 >> v1:172.21.15.3:6789/0 conn(0x7fc1240adcd0 legacy=0x7fc1240cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:32.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc11bfff700 1 --2- 172.21.15.3:0/3197451536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc110005e10 0x7fc110006200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:32.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc11bfff700 1 --2- 172.21.15.3:0/3197451536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc110006a80 0x7fc110006ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:32.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc11bfff700 1 --2- 172.21.15.3:0/3197451536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110007430 0x7fc110010040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:32.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc110010580 con 0x7fc110006a80 2023-03-03T06:25:32.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1100106c0 con 0x7fc110007430 2023-03-03T06:25:32.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc110010800 con 0x7fc110005e10 2023-03-03T06:25:32.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc128d23700 1 --2- 172.21.15.3:0/3197451536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc110006a80 0x7fc110006ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:32.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc129524700 1 --2- 172.21.15.3:0/3197451536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc110005e10 0x7fc110006200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:32.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc129d25700 1 --2- 172.21.15.3:0/3197451536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110007430 0x7fc110010040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:32.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc128d23700 1 -- 172.21.15.3:0/3197451536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc110005e10 msgr2=0x7fc110006200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:32.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc128d23700 1 --2- 172.21.15.3:0/3197451536 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc110005e10 0x7fc110006200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc128d23700 1 -- 172.21.15.3:0/3197451536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110007430 msgr2=0x7fc110010040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:32.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc128d23700 1 --2- 172.21.15.3:0/3197451536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc110007430 0x7fc110010040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc128d23700 1 -- 172.21.15.3:0/3197451536 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1100057e0 con 0x7fc110006a80 2023-03-03T06:25:32.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc129d25700 1 -- 172.21.15.3:0/3197451536 reap_dead start 2023-03-03T06:25:32.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc128d23700 1 --2- 172.21.15.3:0/3197451536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc110006a80 0x7fc110006ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c160 tx=0x7fc110007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19db535fb51095ed server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:32.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400cc60 con 0x7fc110006a80 2023-03-03T06:25:32.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11400cdc0 con 0x7fc110006a80 2023-03-03T06:25:32.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc11bfff700 1 -- 172.21.15.3:0/3197451536 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114005bc0 con 0x7fc110006a80 2023-03-03T06:25:32.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc12a526700 1 -- 172.21.15.3:0/3197451536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc110006a80 msgr2=0x7fc110006ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:32.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.287+0000 7fc12a526700 1 --2- 172.21.15.3:0/3197451536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc110006a80 0x7fc110006ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c160 tx=0x7fc110007ab0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.291+0000 7fc12a526700 1 -- 172.21.15.3:0/3197451536 shutdown_connections 2023-03-03T06:25:32.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.291+0000 7fc12a526700 1 --2- 172.21.15.3:0/3197451536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc110006a80 0x7fc110006ef0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.291+0000 7fc12a526700 1 -- 172.21.15.3:0/3197451536 >> 172.21.15.3:0/3197451536 conn(0x7fc124070a30 msgr2=0x7fc124070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:32.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.291+0000 7fc12a526700 1 -- 172.21.15.3:0/3197451536 shutdown_connections 2023-03-03T06:25:32.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.291+0000 7fc12a526700 1 -- 172.21.15.3:0/3197451536 wait complete. 2023-03-03T06:25:32.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.291+0000 7fc12a526700 1 Processor -- start 2023-03-03T06:25:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc12a526700 1 -- start start 2023-03-03T06:25:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc12a526700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc110006a80 0x7fc1240c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc12a526700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1241561f0 0x7fc1241537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc12a526700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc124156640 0x7fc124153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc12a526700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12415aeb0 con 0x7fc124156640 2023-03-03T06:25:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc12a526700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1240c8d30 con 0x7fc1241561f0 2023-03-03T06:25:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc12a526700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12415a830 con 0x7fc110006a80 2023-03-03T06:25:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc129d25700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc124156640 0x7fc124153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc129d25700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc124156640 0x7fc124153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52816/0 (socket says 172.21.15.3:52816) 2023-03-03T06:25:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc129d25700 1 -- 172.21.15.3:0/3807588647 learned_addr learned my addr 172.21.15.3:0/3807588647 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc129524700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc110006a80 0x7fc1240c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc128d23700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1241561f0 0x7fc1241537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc129d25700 1 -- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc110006a80 msgr2=0x7fc1240c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc129d25700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc110006a80 0x7fc1240c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc129d25700 1 -- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1241561f0 msgr2=0x7fc1241537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc129d25700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1241561f0 0x7fc1241537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc129d25700 1 -- 172.21.15.3:0/3807588647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1240ccb40 con 0x7fc124156640 2023-03-03T06:25:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc129d25700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc124156640 0x7fc124153d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc12406a990 tx=0x7fc11c011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc119ffb700 1 -- 172.21.15.3:0/3807588647 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c00e040 con 0x7fc124156640 2023-03-03T06:25:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc119ffb700 1 -- 172.21.15.3:0/3807588647 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11c00f040 con 0x7fc124156640 2023-03-03T06:25:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.295+0000 7fc119ffb700 1 -- 172.21.15.3:0/3807588647 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c00fa00 con 0x7fc124156640 2023-03-03T06:25:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.299+0000 7fc12a526700 1 -- 172.21.15.3:0/3807588647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1240cc860 con 0x7fc124156640 2023-03-03T06:25:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.299+0000 7fc12a526700 1 -- 172.21.15.3:0/3807588647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc124158e80 con 0x7fc124156640 2023-03-03T06:25:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.299+0000 7fc12a526700 1 -- 172.21.15.3:0/3807588647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12415aff0 con 0x7fc124156640 2023-03-03T06:25:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.303+0000 7fc119ffb700 1 -- 172.21.15.3:0/3807588647 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc11c025070 con 0x7fc124156640 2023-03-03T06:25:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.303+0000 7fc119ffb700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc108064ee0 0x7fc108067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.303+0000 7fc119ffb700 1 -- 172.21.15.3:0/3807588647 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc11c082900 con 0x7fc124156640 2023-03-03T06:25:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.303+0000 7fc129524700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc108064ee0 0x7fc108067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.307+0000 7fc129524700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc108064ee0 0x7fc108067390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc12404f390 tx=0x7fc120009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.311+0000 7fc119ffb700 1 -- 172.21.15.3:0/3807588647 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc12415aff0 con 0x7fc124156640 2023-03-03T06:25:32.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.503+0000 7fc12a526700 1 -- 172.21.15.3:0/3807588647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc12415aff0 con 0x7fc124156640 2023-03-03T06:25:32.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.503+0000 7fc119ffb700 1 -- 172.21.15.3:0/3807588647 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc12415aff0 con 0x7fc124156640 2023-03-03T06:25:32.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.507+0000 7fc12a526700 1 -- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc108064ee0 msgr2=0x7fc108067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:32.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.507+0000 7fc12a526700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc108064ee0 0x7fc108067390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc12404f390 tx=0x7fc120009040 comp rx=0 tx=0).stop 2023-03-03T06:25:32.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.507+0000 7fc12a526700 1 -- 172.21.15.3:0/3807588647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc124156640 msgr2=0x7fc124153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:32.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.507+0000 7fc12a526700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc124156640 0x7fc124153d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc12406a990 tx=0x7fc11c011f80 comp rx=0 tx=0).stop 2023-03-03T06:25:32.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.507+0000 7fc12a526700 1 -- 172.21.15.3:0/3807588647 shutdown_connections 2023-03-03T06:25:32.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.507+0000 7fc12a526700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc108064ee0 0x7fc108067390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.507+0000 7fc12a526700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc124156640 0x7fc124153d10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.507+0000 7fc12a526700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1241561f0 0x7fc1241537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.507+0000 7fc12a526700 1 --2- 172.21.15.3:0/3807588647 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc110006a80 0x7fc1240c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:32.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.507+0000 7fc12a526700 1 -- 172.21.15.3:0/3807588647 >> 172.21.15.3:0/3807588647 conn(0x7fc124070a30 msgr2=0x7fc124070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:32.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.507+0000 7fc12a526700 1 -- 172.21.15.3:0/3807588647 shutdown_connections 2023-03-03T06:25:32.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:32.507+0000 7fc12a526700 1 -- 172.21.15.3:0/3807588647 wait complete. 2023-03-03T06:25:32.513 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2023-03-03T06:25:32.546 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2023-03-03T06:25:32.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:32.547 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-03-03T06:25:32.592 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[37589]: fuse finished with error 0 and tester_r 0 2023-03-03T06:25:32.600 INFO:teuthology.orchestra.run:waiting for 300 2023-03-03T06:25:38.701 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-03-03T06:25:38.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:38.703 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-03-03T06:25:38.742 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:25:38.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.991+0000 7f7d35e1e700 1 Processor -- start 2023-03-03T06:25:38.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.991+0000 7f7d35e1e700 1 -- start start 2023-03-03T06:25:38.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d35e1e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7d3015c750 0x7f7d30158440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:38.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d35e1e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d300aee70 0x7f7d300ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:38.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d35e1e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d300cdd30 con 0x7f7d3006a780 2023-03-03T06:25:38.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d35e1e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d30158980 con 0x7f7d3015c750 2023-03-03T06:25:38.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d35e1e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d300ca4d0 con 0x7f7d300aee70 2023-03-03T06:25:38.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d2ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7d3015c750 0x7f7d30158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:38.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d3561d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d300aee70 0x7f7d300ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:38.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d2ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7d3015c750 0x7f7d30158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52822/0 (socket says 172.21.15.3:52822) 2023-03-03T06:25:38.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d2ffff700 1 -- 172.21.15.3:0/155852768 learned_addr learned my addr 172.21.15.3:0/155852768 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:38.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d34e1c700 1 --1- 172.21.15.3:0/155852768 >> v1:172.21.15.32:6789/0 conn(0x7f7d3006a780 0x7f7d3015c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:33776/0 (socket says 172.21.15.3:33776) 2023-03-03T06:25:39.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d3561d700 1 -- 172.21.15.3:0/155852768 >> v2:172.21.15.3:3300/0 conn(0x7f7d3015c750 msgr2=0x7f7d30158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:39.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d3561d700 1 --2- 172.21.15.3:0/155852768 >> v2:172.21.15.3:3300/0 conn(0x7f7d3015c750 0x7f7d30158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d3561d700 1 -- 172.21.15.3:0/155852768 >> v1:172.21.15.32:6789/0 conn(0x7f7d3006a780 legacy=0x7f7d3015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d3561d700 1 -- 172.21.15.3:0/155852768 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d300cd6e0 con 0x7f7d300aee70 2023-03-03T06:25:39.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d2ffff700 1 --2- 172.21.15.3:0/155852768 >> v2:172.21.15.3:3300/0 conn(0x7f7d3015c750 0x7f7d30158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:39.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d3561d700 1 --2- 172.21.15.3:0/155852768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d300aee70 0x7f7d300ce1c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d300ccc20 tx=0x7f7d2800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60bfd3fe20efece8 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:39.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.995+0000 7f7d2f7fe700 1 -- 172.21.15.3:0/155852768 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800e070 con 0x7f7d300aee70 2023-03-03T06:25:39.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.999+0000 7f7d2f7fe700 1 -- 172.21.15.3:0/155852768 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2800bb20 con 0x7f7d300aee70 2023-03-03T06:25:39.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.999+0000 7f7d2f7fe700 1 -- 172.21.15.3:0/155852768 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28012430 con 0x7f7d300aee70 2023-03-03T06:25:39.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.999+0000 7f7d35e1e700 1 -- 172.21.15.3:0/155852768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d300aee70 msgr2=0x7f7d300ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.999+0000 7f7d35e1e700 1 --2- 172.21.15.3:0/155852768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d300aee70 0x7f7d300ce1c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d300ccc20 tx=0x7f7d2800b330 comp rx=0 tx=0).stop 2023-03-03T06:25:39.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.999+0000 7f7d35e1e700 1 -- 172.21.15.3:0/155852768 shutdown_connections 2023-03-03T06:25:39.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.999+0000 7f7d35e1e700 1 --2- 172.21.15.3:0/155852768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d300aee70 0x7f7d300ce1c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.999+0000 7f7d35e1e700 1 --2- 172.21.15.3:0/155852768 >> v2:172.21.15.3:3300/0 conn(0x7f7d3015c750 0x7f7d30158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.999+0000 7f7d35e1e700 1 -- 172.21.15.3:0/155852768 >> 172.21.15.3:0/155852768 conn(0x7f7d30070a30 msgr2=0x7f7d30070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:39.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.999+0000 7f7d35e1e700 1 -- 172.21.15.3:0/155852768 shutdown_connections 2023-03-03T06:25:39.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:38.999+0000 7f7d35e1e700 1 -- 172.21.15.3:0/155852768 wait complete. 2023-03-03T06:25:39.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d35e1e700 1 Processor -- start 2023-03-03T06:25:39.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d35e1e700 1 -- start start 2023-03-03T06:25:39.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d35e1e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d3006a780 0x7f7d30158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d35e1e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d300aee70 0x7f7d300c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d35e1e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d3015c750 0x7f7d30142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d35e1e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3014a060 con 0x7f7d3015c750 2023-03-03T06:25:39.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d35e1e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300c4d90 con 0x7f7d3006a780 2023-03-03T06:25:39.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d35e1e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300c4fe0 con 0x7f7d300aee70 2023-03-03T06:25:39.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d3561d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d3015c750 0x7f7d30142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d3561d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d3015c750 0x7f7d30142c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52828/0 (socket says 172.21.15.3:52828) 2023-03-03T06:25:39.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d3561d700 1 -- 172.21.15.3:0/1005918111 learned_addr learned my addr 172.21.15.3:0/1005918111 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:39.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d34e1c700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d3006a780 0x7f7d30158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d2ffff700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d300aee70 0x7f7d300c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d3561d700 1 -- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d300aee70 msgr2=0x7f7d300c12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d3561d700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d300aee70 0x7f7d300c12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d3561d700 1 -- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d3006a780 msgr2=0x7f7d30158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d3561d700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d3006a780 0x7f7d30158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.003+0000 7f7d3561d700 1 -- 172.21.15.3:0/1005918111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c009570 con 0x7f7d3015c750 2023-03-03T06:25:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.007+0000 7f7d3561d700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d3015c750 0x7f7d30142c10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7d28014040 tx=0x7f7d28004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.007+0000 7f7d2d7fa700 1 -- 172.21.15.3:0/1005918111 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28012430 con 0x7f7d3015c750 2023-03-03T06:25:39.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.007+0000 7f7d2d7fa700 1 -- 172.21.15.3:0/1005918111 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d28020070 con 0x7f7d3015c750 2023-03-03T06:25:39.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.007+0000 7f7d35e1e700 1 -- 172.21.15.3:0/1005918111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d28009a70 con 0x7f7d3015c750 2023-03-03T06:25:39.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.007+0000 7f7d2d7fa700 1 -- 172.21.15.3:0/1005918111 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28004280 con 0x7f7d3015c750 2023-03-03T06:25:39.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.007+0000 7f7d35e1e700 1 -- 172.21.15.3:0/1005918111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d300ca7b0 con 0x7f7d3015c750 2023-03-03T06:25:39.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.007+0000 7f7d35e1e700 1 -- 172.21.15.3:0/1005918111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d300cd9c0 con 0x7f7d3015c750 2023-03-03T06:25:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.011+0000 7f7d2d7fa700 1 -- 172.21.15.3:0/1005918111 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7d2802e070 con 0x7f7d3015c750 2023-03-03T06:25:39.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.011+0000 7f7d2d7fa700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d14064ee0 0x7f7d14067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.011+0000 7f7d2d7fa700 1 -- 172.21.15.3:0/1005918111 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7d28022070 con 0x7f7d3015c750 2023-03-03T06:25:39.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.015+0000 7f7d34e1c700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d14064ee0 0x7f7d14067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.015+0000 7f7d34e1c700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d14064ee0 0x7f7d14067390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d24000f60 tx=0x7f7d24007ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:39.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.019+0000 7f7d2d7fa700 1 -- 172.21.15.3:0/1005918111 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7d28051cb0 con 0x7f7d3015c750 2023-03-03T06:25:39.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.195+0000 7f7d35e1e700 1 -- 172.21.15.3:0/1005918111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d3015cd90 con 0x7f7d3015c750 2023-03-03T06:25:39.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.199+0000 7f7d2d7fa700 1 -- 172.21.15.3:0/1005918111 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19556 (secure 0 0 0) 0x7f7d2802c090 con 0x7f7d3015c750 2023-03-03T06:25:39.203 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:39.204 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:18.863350+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:25:39.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.203+0000 7f7d35e1e700 1 -- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d14064ee0 msgr2=0x7f7d14067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.203+0000 7f7d35e1e700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d14064ee0 0x7f7d14067390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d24000f60 tx=0x7f7d24007ef0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.203+0000 7f7d35e1e700 1 -- 172.21.15.3:0/1005918111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d3015c750 msgr2=0x7f7d30142c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.203+0000 7f7d35e1e700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d3015c750 0x7f7d30142c10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7d28014040 tx=0x7f7d28004af0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.203+0000 7f7d35e1e700 1 -- 172.21.15.3:0/1005918111 shutdown_connections 2023-03-03T06:25:39.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.203+0000 7f7d35e1e700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d14064ee0 0x7f7d14067390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.203+0000 7f7d35e1e700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d3015c750 0x7f7d30142c10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.203+0000 7f7d35e1e700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d300aee70 0x7f7d300c12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.203+0000 7f7d35e1e700 1 --2- 172.21.15.3:0/1005918111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d3006a780 0x7f7d30158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.203+0000 7f7d35e1e700 1 -- 172.21.15.3:0/1005918111 >> 172.21.15.3:0/1005918111 conn(0x7f7d30070a30 msgr2=0x7f7d300caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:39.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.207+0000 7f7d35e1e700 1 -- 172.21.15.3:0/1005918111 shutdown_connections 2023-03-03T06:25:39.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.207+0000 7f7d35e1e700 1 -- 172.21.15.3:0/1005918111 wait complete. 2023-03-03T06:25:39.243 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:25:39.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.443+0000 7ff492ecb700 1 Processor -- start 2023-03-03T06:25:39.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.443+0000 7ff492ecb700 1 -- start start 2023-03-03T06:25:39.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.443+0000 7ff492ecb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff48c159100 0x7ff48c0ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.443+0000 7ff492ecb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff48c0cde60 0x7ff48c0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.443+0000 7ff492ecb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff48c0ce550 0x7ff48c0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.443+0000 7ff492ecb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48c0cb010 con 0x7ff48c0cde60 2023-03-03T06:25:39.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.443+0000 7ff492ecb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48c0cef10 con 0x7ff48c0ce550 2023-03-03T06:25:39.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.443+0000 7ff492ecb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0ae710 con 0x7ff48c159100 2023-03-03T06:25:39.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff4916c8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff48c0cde60 0x7ff48c0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff4916c8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff48c0cde60 0x7ff48c0afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52834/0 (socket says 172.21.15.3:52834) 2023-03-03T06:25:39.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff4916c8700 1 -- 172.21.15.3:0/3513641616 learned_addr learned my addr 172.21.15.3:0/3513641616 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:39.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff4926ca700 1 --2- 172.21.15.3:0/3513641616 >> v2:172.21.15.32:3300/0 conn(0x7ff48c0ce550 0x7ff48c0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff4916c8700 1 -- 172.21.15.3:0/3513641616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff48c159100 msgr2=0x7ff48c0ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:39.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff4916c8700 1 --2- 172.21.15.3:0/3513641616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff48c159100 0x7ff48c0ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff4916c8700 1 -- 172.21.15.3:0/3513641616 >> v2:172.21.15.32:3300/0 conn(0x7ff48c0ce550 msgr2=0x7ff48c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:39.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff4916c8700 1 --2- 172.21.15.3:0/3513641616 >> v2:172.21.15.32:3300/0 conn(0x7ff48c0ce550 0x7ff48c0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff4916c8700 1 -- 172.21.15.3:0/3513641616 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c15d8a0 con 0x7ff48c0cde60 2023-03-03T06:25:39.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff4926ca700 1 --2- 172.21.15.3:0/3513641616 >> v2:172.21.15.32:3300/0 conn(0x7ff48c0ce550 0x7ff48c0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:25:39.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff4916c8700 1 --2- 172.21.15.3:0/3513641616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff48c0cde60 0x7ff48c0afb60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0ade10 tx=0x7ff47c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02e3094d039b01a server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:39.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff490ec7700 1 -- 172.21.15.3:0/3513641616 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00b8a0 con 0x7ff48c0cde60 2023-03-03T06:25:39.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff490ec7700 1 -- 172.21.15.3:0/3513641616 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c00ba00 con 0x7ff48c0cde60 2023-03-03T06:25:39.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff490ec7700 1 -- 172.21.15.3:0/3513641616 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00bd90 con 0x7ff48c0cde60 2023-03-03T06:25:39.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff492ecb700 1 -- 172.21.15.3:0/3513641616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff48c0cde60 msgr2=0x7ff48c0afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff492ecb700 1 --2- 172.21.15.3:0/3513641616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff48c0cde60 0x7ff48c0afb60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0ade10 tx=0x7ff47c009bf0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff492ecb700 1 -- 172.21.15.3:0/3513641616 shutdown_connections 2023-03-03T06:25:39.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff492ecb700 1 --2- 172.21.15.3:0/3513641616 >> v2:172.21.15.32:3300/0 conn(0x7ff48c0ce550 0x7ff48c0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff492ecb700 1 --2- 172.21.15.3:0/3513641616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff48c0cde60 0x7ff48c0afb60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff492ecb700 1 --2- 172.21.15.3:0/3513641616 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff48c159100 0x7ff48c0ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.447+0000 7ff492ecb700 1 -- 172.21.15.3:0/3513641616 >> 172.21.15.3:0/3513641616 conn(0x7ff48c070a30 msgr2=0x7ff48c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:39.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.451+0000 7ff492ecb700 1 -- 172.21.15.3:0/3513641616 shutdown_connections 2023-03-03T06:25:39.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.451+0000 7ff492ecb700 1 -- 172.21.15.3:0/3513641616 wait complete. 2023-03-03T06:25:39.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.451+0000 7ff492ecb700 1 Processor -- start 2023-03-03T06:25:39.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.451+0000 7ff492ecb700 1 -- start start 2023-03-03T06:25:39.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff492ecb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff48c0cde60 0x7ff48c14f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff492ecb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff48c0ce550 0x7ff48c0c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff492ecb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff48c159100 0x7ff48c156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff492ecb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c150630 con 0x7ff48c0ce550 2023-03-03T06:25:39.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff492ecb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0c3320 con 0x7ff48c159100 2023-03-03T06:25:39.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff492ecb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c14fd90 con 0x7ff48c0cde60 2023-03-03T06:25:39.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4916c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff48c0ce550 0x7ff48c0c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4916c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff48c0ce550 0x7ff48c0c2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52844/0 (socket says 172.21.15.3:52844) 2023-03-03T06:25:39.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4916c8700 1 -- 172.21.15.3:0/624304941 learned_addr learned my addr 172.21.15.3:0/624304941 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:39.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4926ca700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff48c159100 0x7ff48c156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff491ec9700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff48c0cde60 0x7ff48c14f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4916c8700 1 -- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff48c0cde60 msgr2=0x7ff48c14f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4916c8700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff48c0cde60 0x7ff48c14f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4916c8700 1 -- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff48c159100 msgr2=0x7ff48c156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4916c8700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff48c159100 0x7ff48c156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4916c8700 1 -- 172.21.15.3:0/624304941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c009570 con 0x7ff48c0ce550 2023-03-03T06:25:39.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4926ca700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff48c159100 0x7ff48c156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:39.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4916c8700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff48c0ce550 0x7ff48c0c2de0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff47c000c00 tx=0x7ff47c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:39.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4827fc700 1 -- 172.21.15.3:0/624304941 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c003ad0 con 0x7ff48c0ce550 2023-03-03T06:25:39.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4827fc700 1 -- 172.21.15.3:0/624304941 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c01e070 con 0x7ff48c0ce550 2023-03-03T06:25:39.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff4827fc700 1 -- 172.21.15.3:0/624304941 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c003e70 con 0x7ff48c0ce550 2023-03-03T06:25:39.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff492ecb700 1 -- 172.21.15.3:0/624304941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c0cad30 con 0x7ff48c0ce550 2023-03-03T06:25:39.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.455+0000 7ff492ecb700 1 -- 172.21.15.3:0/624304941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c0caa50 con 0x7ff48c0ce550 2023-03-03T06:25:39.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.459+0000 7ff492ecb700 1 -- 172.21.15.3:0/624304941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48c15db60 con 0x7ff48c0ce550 2023-03-03T06:25:39.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.463+0000 7ff4827fc700 1 -- 172.21.15.3:0/624304941 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff47c00f050 con 0x7ff48c0ce550 2023-03-03T06:25:39.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.463+0000 7ff4827fc700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff470064ee0 0x7ff470067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.463+0000 7ff4827fc700 1 -- 172.21.15.3:0/624304941 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff47c020070 con 0x7ff48c0ce550 2023-03-03T06:25:39.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.467+0000 7ff491ec9700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff470064ee0 0x7ff470067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.467+0000 7ff491ec9700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff470064ee0 0x7ff470067390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff488006fd0 tx=0x7ff488008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:39.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.471+0000 7ff4827fc700 1 -- 172.21.15.3:0/624304941 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff47c051850 con 0x7ff48c0ce550 2023-03-03T06:25:39.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.643+0000 7ff492ecb700 1 -- 172.21.15.3:0/624304941 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff48c150470 con 0x7ff48c0ce550 2023-03-03T06:25:39.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.647+0000 7ff4827fc700 1 -- 172.21.15.3:0/624304941 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19556 (secure 0 0 0) 0x7ff47c054e70 con 0x7ff48c0ce550 2023-03-03T06:25:39.651 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:39.651 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:18.863350+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:25:39.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.647+0000 7ff492ecb700 1 -- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff470064ee0 msgr2=0x7ff470067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.647+0000 7ff492ecb700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff470064ee0 0x7ff470067390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff488006fd0 tx=0x7ff488008040 comp rx=0 tx=0).stop 2023-03-03T06:25:39.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.647+0000 7ff492ecb700 1 -- 172.21.15.3:0/624304941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff48c0ce550 msgr2=0x7ff48c0c2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.647+0000 7ff492ecb700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff48c0ce550 0x7ff48c0c2de0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff47c000c00 tx=0x7ff47c004770 comp rx=0 tx=0).stop 2023-03-03T06:25:39.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.647+0000 7ff492ecb700 1 -- 172.21.15.3:0/624304941 shutdown_connections 2023-03-03T06:25:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.647+0000 7ff492ecb700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff470064ee0 0x7ff470067390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.647+0000 7ff492ecb700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff48c159100 0x7ff48c156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.647+0000 7ff492ecb700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff48c0ce550 0x7ff48c0c2de0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.647+0000 7ff492ecb700 1 --2- 172.21.15.3:0/624304941 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff48c0cde60 0x7ff48c14f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.647+0000 7ff492ecb700 1 -- 172.21.15.3:0/624304941 >> 172.21.15.3:0/624304941 conn(0x7ff48c070a30 msgr2=0x7ff48c06e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:39.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.651+0000 7ff492ecb700 1 -- 172.21.15.3:0/624304941 shutdown_connections 2023-03-03T06:25:39.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.651+0000 7ff492ecb700 1 -- 172.21.15.3:0/624304941 wait complete. 2023-03-03T06:25:39.687 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-03-03T06:25:39.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.895+0000 7f837c866700 1 Processor -- start 2023-03-03T06:25:39.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.895+0000 7f837c866700 1 -- start start 2023-03-03T06:25:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f837c866700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8378156ba0 0x7f8378071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f837c866700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83780d0b30 0x7f83780d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f837c866700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83780cf3b0 con 0x7f837815d6e0 2023-03-03T06:25:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f837c866700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83780d14f0 con 0x7f8378156ba0 2023-03-03T06:25:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f837c866700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8378156f70 con 0x7f83780d0b30 2023-03-03T06:25:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f83777fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8378156ba0 0x7f8378071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f83777fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8378156ba0 0x7f8378071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52860/0 (socket says 172.21.15.3:52860) 2023-03-03T06:25:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f83777fe700 1 -- 172.21.15.3:0/3644396443 learned_addr learned my addr 172.21.15.3:0/3644396443 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f8377fff700 1 --2- 172.21.15.3:0/3644396443 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83780d0b30 0x7f83780d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f83777fe700 1 -- 172.21.15.3:0/3644396443 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83780d0b30 msgr2=0x7f83780d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f83777fe700 1 --2- 172.21.15.3:0/3644396443 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83780d0b30 0x7f83780d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f83777fe700 1 -- 172.21.15.3:0/3644396443 >> v1:172.21.15.3:6789/0 conn(0x7f837815d6e0 legacy=0x7f83780cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:39.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f83777fe700 1 -- 172.21.15.3:0/3644396443 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83780cd320 con 0x7f8378156ba0 2023-03-03T06:25:39.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f83777fe700 1 --2- 172.21.15.3:0/3644396443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8378156ba0 0x7f8378071280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8368009910 tx=0x7f8368009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ce384028670748 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f83767fc700 1 -- 172.21.15.3:0/3644396443 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368004230 con 0x7f8378156ba0 2023-03-03T06:25:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f83767fc700 1 -- 172.21.15.3:0/3644396443 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8368004390 con 0x7f8378156ba0 2023-03-03T06:25:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.899+0000 7f83767fc700 1 -- 172.21.15.3:0/3644396443 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368009e60 con 0x7f8378156ba0 2023-03-03T06:25:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.903+0000 7f837c866700 1 -- 172.21.15.3:0/3644396443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8378156ba0 msgr2=0x7f8378071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.903+0000 7f837c866700 1 --2- 172.21.15.3:0/3644396443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8378156ba0 0x7f8378071280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8368009910 tx=0x7f8368009c20 comp rx=0 tx=0).stop 2023-03-03T06:25:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.903+0000 7f837c866700 1 -- 172.21.15.3:0/3644396443 shutdown_connections 2023-03-03T06:25:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.903+0000 7f837c866700 1 --2- 172.21.15.3:0/3644396443 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f83780d0b30 0x7f83780d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.903+0000 7f837c866700 1 --2- 172.21.15.3:0/3644396443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8378156ba0 0x7f8378071280 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.903+0000 7f837c866700 1 -- 172.21.15.3:0/3644396443 >> 172.21.15.3:0/3644396443 conn(0x7f837806e5c0 msgr2=0x7f837806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.903+0000 7f837c866700 1 -- 172.21.15.3:0/3644396443 shutdown_connections 2023-03-03T06:25:39.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.903+0000 7f837c866700 1 -- 172.21.15.3:0/3644396443 wait complete. 2023-03-03T06:25:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.903+0000 7f837c866700 1 Processor -- start 2023-03-03T06:25:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f837c866700 1 -- start start 2023-03-03T06:25:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f837c866700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83780d0b30 0x7f837814ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f837c866700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8378156ba0 0x7f83780c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f837c866700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f837815d6e0 0x7f837815c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f837c866700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83780b3420 con 0x7f83780d0b30 2023-03-03T06:25:39.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f837c866700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83780c9170 con 0x7f837815d6e0 2023-03-03T06:25:39.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f837c866700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83780b2810 con 0x7f8378156ba0 2023-03-03T06:25:39.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f83777fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83780d0b30 0x7f837814ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f83777fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83780d0b30 0x7f837814ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52862/0 (socket says 172.21.15.3:52862) 2023-03-03T06:25:39.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f83777fe700 1 -- 172.21.15.3:0/2020725280 learned_addr learned my addr 172.21.15.3:0/2020725280 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:39.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f8377fff700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f837815d6e0 0x7f837815c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f8376ffd700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8378156ba0 0x7f83780c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f83777fe700 1 -- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8378156ba0 msgr2=0x7f83780c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f83777fe700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8378156ba0 0x7f83780c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f83777fe700 1 -- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f837815d6e0 msgr2=0x7f837815c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:39.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f83777fe700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f837815d6e0 0x7f837815c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:39.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f83777fe700 1 -- 172.21.15.3:0/2020725280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8368009570 con 0x7f83780d0b30 2023-03-03T06:25:39.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.907+0000 7f8376ffd700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8378156ba0 0x7f83780c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:25:39.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.911+0000 7f83777fe700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83780d0b30 0x7f837814ec40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8368004000 tx=0x7f8368004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:39.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.911+0000 7f835ffff700 1 -- 172.21.15.3:0/2020725280 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83680036c0 con 0x7f83780d0b30 2023-03-03T06:25:39.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.911+0000 7f835ffff700 1 -- 172.21.15.3:0/2020725280 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8368003820 con 0x7f83780d0b30 2023-03-03T06:25:39.920 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.911+0000 7f835ffff700 1 -- 172.21.15.3:0/2020725280 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368003bd0 con 0x7f83780d0b30 2023-03-03T06:25:39.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.911+0000 7f837c866700 1 -- 172.21.15.3:0/2020725280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83780cd8e0 con 0x7f83780d0b30 2023-03-03T06:25:39.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.911+0000 7f837c866700 1 -- 172.21.15.3:0/2020725280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83780cd600 con 0x7f83780d0b30 2023-03-03T06:25:39.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.911+0000 7f837c866700 1 -- 172.21.15.3:0/2020725280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83780717c0 con 0x7f83780d0b30 2023-03-03T06:25:39.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.915+0000 7f835ffff700 1 -- 172.21.15.3:0/2020725280 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8368017020 con 0x7f83780d0b30 2023-03-03T06:25:39.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.915+0000 7f835ffff700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8364064ee0 0x7f8364067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:39.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.915+0000 7f835ffff700 1 -- 172.21.15.3:0/2020725280 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f83680836f0 con 0x7f83780d0b30 2023-03-03T06:25:39.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.915+0000 7f8376ffd700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8364064ee0 0x7f8364067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:39.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.919+0000 7f8376ffd700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8364064ee0 0x7f8364067390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f837815a400 tx=0x7f836c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:39.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:39.923+0000 7f835ffff700 1 -- 172.21.15.3:0/2020725280 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f8368051450 con 0x7f83780d0b30 2023-03-03T06:25:40.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.143+0000 7f837c866700 1 -- 172.21.15.3:0/2020725280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8378073e90 con 0x7f83780d0b30 2023-03-03T06:25:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.755+0000 7f835ffff700 1 -- 172.21.15.3:0/2020725280 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8368022360 con 0x7f83780d0b30 2023-03-03T06:25:40.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.759+0000 7f837c866700 1 -- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8364064ee0 msgr2=0x7f8364067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:40.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.759+0000 7f837c866700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8364064ee0 0x7f8364067390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f837815a400 tx=0x7f836c009040 comp rx=0 tx=0).stop 2023-03-03T06:25:40.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.759+0000 7f837c866700 1 -- 172.21.15.3:0/2020725280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83780d0b30 msgr2=0x7f837814ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:40.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.759+0000 7f837c866700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83780d0b30 0x7f837814ec40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8368004000 tx=0x7f8368004770 comp rx=0 tx=0).stop 2023-03-03T06:25:40.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.759+0000 7f837c866700 1 -- 172.21.15.3:0/2020725280 shutdown_connections 2023-03-03T06:25:40.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.759+0000 7f837c866700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8364064ee0 0x7f8364067390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:40.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.759+0000 7f837c866700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f837815d6e0 0x7f837815c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:40.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.759+0000 7f837c866700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8378156ba0 0x7f83780c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:40.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.759+0000 7f837c866700 1 --2- 172.21.15.3:0/2020725280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f83780d0b30 0x7f837814ec40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:40.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.759+0000 7f837c866700 1 -- 172.21.15.3:0/2020725280 >> 172.21.15.3:0/2020725280 conn(0x7f837806e5c0 msgr2=0x7f83780cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:40.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.759+0000 7f837c866700 1 -- 172.21.15.3:0/2020725280 shutdown_connections 2023-03-03T06:25:40.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:40.759+0000 7f837c866700 1 -- 172.21.15.3:0/2020725280 wait complete. 2023-03-03T06:25:40.801 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-03-03T06:25:40.802 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-03-03T06:25:41.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.007+0000 7f63988b1700 1 Processor -- start 2023-03-03T06:25:41.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.007+0000 7f63988b1700 1 -- start start 2023-03-03T06:25:41.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63988b1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63940b07c0 0x7f63940b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63988b1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63940b1850 0x7f63940b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63988b1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639406d750 con 0x7f63940caab0 2023-03-03T06:25:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63988b1700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63940c9510 con 0x7f63940b07c0 2023-03-03T06:25:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63988b1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639415a640 con 0x7f63940b1850 2023-03-03T06:25:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63937fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63940b07c0 0x7f63940b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63937fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f63940b07c0 0x7f63940b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43210/0 (socket says 172.21.15.3:43210) 2023-03-03T06:25:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63937fe700 1 -- 172.21.15.3:0/1142205773 learned_addr learned my addr 172.21.15.3:0/1142205773 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63937fe700 1 -- 172.21.15.3:0/1142205773 >> v2:172.21.15.32:3300/0 conn(0x7f63940b1850 msgr2=0x7f63940b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63937fe700 1 --2- 172.21.15.3:0/1142205773 >> v2:172.21.15.32:3300/0 conn(0x7f63940b1850 0x7f63940b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63937fe700 1 -- 172.21.15.3:0/1142205773 >> v1:172.21.15.3:6789/0 conn(0x7f63940caab0 legacy=0x7f63940c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63937fe700 1 -- 172.21.15.3:0/1142205773 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940b10d0 con 0x7f63940b07c0 2023-03-03T06:25:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63937fe700 1 --2- 172.21.15.3:0/1142205773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b07c0 0x7f63940b0b90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f63940c8d90 tx=0x7f6384009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d086a5712d9912b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f6392ffd700 1 -- 172.21.15.3:0/1142205773 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638400b8a0 con 0x7f63940b07c0 2023-03-03T06:25:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f6392ffd700 1 -- 172.21.15.3:0/1142205773 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638400ba00 con 0x7f63940b07c0 2023-03-03T06:25:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f6392ffd700 1 -- 172.21.15.3:0/1142205773 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638400bd90 con 0x7f63940b07c0 2023-03-03T06:25:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63988b1700 1 -- 172.21.15.3:0/1142205773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b07c0 msgr2=0x7f63940b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.011+0000 7f63988b1700 1 --2- 172.21.15.3:0/1142205773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b07c0 0x7f63940b0b90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f63940c8d90 tx=0x7f6384009bf0 comp rx=0 tx=0).stop 2023-03-03T06:25:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.015+0000 7f63988b1700 1 -- 172.21.15.3:0/1142205773 shutdown_connections 2023-03-03T06:25:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.015+0000 7f63988b1700 1 --2- 172.21.15.3:0/1142205773 >> v2:172.21.15.32:3300/0 conn(0x7f63940b1850 0x7f63940b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.015+0000 7f63988b1700 1 --2- 172.21.15.3:0/1142205773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b07c0 0x7f63940b0b90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.015+0000 7f63988b1700 1 -- 172.21.15.3:0/1142205773 >> 172.21.15.3:0/1142205773 conn(0x7f63940607c0 msgr2=0x7f6394060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.015+0000 7f63988b1700 1 -- 172.21.15.3:0/1142205773 shutdown_connections 2023-03-03T06:25:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.015+0000 7f63988b1700 1 -- 172.21.15.3:0/1142205773 wait complete. 2023-03-03T06:25:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.015+0000 7f63988b1700 1 Processor -- start 2023-03-03T06:25:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.015+0000 7f63988b1700 1 -- start start 2023-03-03T06:25:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63988b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b07c0 0x7f63940c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63988b1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940b1850 0x7f63941f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63988b1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f63940caab0 0x7f639415b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63988b1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940ae500 con 0x7f63940b07c0 2023-03-03T06:25:41.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63988b1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63941f61c0 con 0x7f63940b1850 2023-03-03T06:25:41.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63988b1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639415add0 con 0x7f63940caab0 2023-03-03T06:25:41.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63937fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b07c0 0x7f63940c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:41.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63937fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b07c0 0x7f63940c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43212/0 (socket says 172.21.15.3:43212) 2023-03-03T06:25:41.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63937fe700 1 -- 172.21.15.3:0/3360036166 learned_addr learned my addr 172.21.15.3:0/3360036166 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:41.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f638affd700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940b1850 0x7f63941f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:41.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63937fe700 1 -- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f63940caab0 msgr2=0x7f639415b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63937fe700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f63940caab0 0x7f639415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63937fe700 1 -- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940b1850 msgr2=0x7f63941f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63937fe700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940b1850 0x7f63941f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63937fe700 1 -- 172.21.15.3:0/3360036166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6384009570 con 0x7f63940b07c0 2023-03-03T06:25:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f638affd700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940b1850 0x7f63941f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:25:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63937fe700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b07c0 0x7f63940c67d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f639406a910 tx=0x7f6384004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f6390ff9700 1 -- 172.21.15.3:0/3360036166 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6384003ad0 con 0x7f63940b07c0 2023-03-03T06:25:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f6390ff9700 1 -- 172.21.15.3:0/3360036166 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638401f070 con 0x7f63940b07c0 2023-03-03T06:25:41.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f6390ff9700 1 -- 172.21.15.3:0/3360036166 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6384003e70 con 0x7f63940b07c0 2023-03-03T06:25:41.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63988b1700 1 -- 172.21.15.3:0/3360036166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639406ce20 con 0x7f63940b07c0 2023-03-03T06:25:41.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.019+0000 7f63988b1700 1 -- 172.21.15.3:0/3360036166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639406cb40 con 0x7f63940b07c0 2023-03-03T06:25:41.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.023+0000 7f63988b1700 1 -- 172.21.15.3:0/3360036166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639406d100 con 0x7f63940b07c0 2023-03-03T06:25:41.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.023+0000 7f6390ff9700 1 -- 172.21.15.3:0/3360036166 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f638400f050 con 0x7f63940b07c0 2023-03-03T06:25:41.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.027+0000 7f6390ff9700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6374064ee0 0x7f6374067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:41.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.027+0000 7f6390ff9700 1 -- 172.21.15.3:0/3360036166 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6384021070 con 0x7f63940b07c0 2023-03-03T06:25:41.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.027+0000 7f638affd700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6374064ee0 0x7f6374067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:41.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.027+0000 7f638affd700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6374064ee0 0x7f6374067390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f639415d850 tx=0x7f637c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:41.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.031+0000 7f6390ff9700 1 -- 172.21.15.3:0/3360036166 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6384051c80 con 0x7f63940b07c0 2023-03-03T06:25:41.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.251+0000 7f63988b1700 1 -- 172.21.15.3:0/3360036166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6394073e90 con 0x7f63940b07c0 2023-03-03T06:25:41.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.763+0000 7f6390ff9700 1 -- 172.21.15.3:0/3360036166 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f63840552a0 con 0x7f63940b07c0 2023-03-03T06:25:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.767+0000 7f63988b1700 1 -- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6374064ee0 msgr2=0x7f6374067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.767+0000 7f63988b1700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6374064ee0 0x7f6374067390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f639415d850 tx=0x7f637c009040 comp rx=0 tx=0).stop 2023-03-03T06:25:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.767+0000 7f63988b1700 1 -- 172.21.15.3:0/3360036166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b07c0 msgr2=0x7f63940c67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.767+0000 7f63988b1700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b07c0 0x7f63940c67d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f639406a910 tx=0x7f6384004770 comp rx=0 tx=0).stop 2023-03-03T06:25:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.767+0000 7f63988b1700 1 -- 172.21.15.3:0/3360036166 shutdown_connections 2023-03-03T06:25:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.767+0000 7f63988b1700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6374064ee0 0x7f6374067390 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.767+0000 7f63988b1700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f63940caab0 0x7f639415b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.767+0000 7f63988b1700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940b1850 0x7f63941f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.767+0000 7f63988b1700 1 --2- 172.21.15.3:0/3360036166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63940b07c0 0x7f63940c67d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:41.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.767+0000 7f63988b1700 1 -- 172.21.15.3:0/3360036166 >> 172.21.15.3:0/3360036166 conn(0x7f63940607c0 msgr2=0x7f639406bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:41.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.771+0000 7f63988b1700 1 -- 172.21.15.3:0/3360036166 shutdown_connections 2023-03-03T06:25:41.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:41.771+0000 7f63988b1700 1 -- 172.21.15.3:0/3360036166 wait complete. 2023-03-03T06:25:41.810 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-03-03T06:25:41.810 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-03T06:25:42.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.019+0000 7f754eda8700 1 Processor -- start 2023-03-03T06:25:42.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.019+0000 7f754eda8700 1 -- start start 2023-03-03T06:25:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.019+0000 7f754eda8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7548160880 0x7f75480c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.019+0000 7f754eda8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75480aea70 0x7f75480ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.019+0000 7f754eda8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75480ce190 con 0x7f75480adcf0 2023-03-03T06:25:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.019+0000 7f754eda8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75480ae190 con 0x7f7548160880 2023-03-03T06:25:42.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.019+0000 7f754eda8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75480c99d0 con 0x7f75480aea70 2023-03-03T06:25:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754dda6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7548160880 0x7f75480c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754dda6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7548160880 0x7f75480c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43222/0 (socket says 172.21.15.3:43222) 2023-03-03T06:25:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754dda6700 1 -- 172.21.15.3:0/2770813586 learned_addr learned my addr 172.21.15.3:0/2770813586 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754dda6700 1 -- 172.21.15.3:0/2770813586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75480aea70 msgr2=0x7f75480ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754dda6700 1 --2- 172.21.15.3:0/2770813586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75480aea70 0x7f75480ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754dda6700 1 -- 172.21.15.3:0/2770813586 >> v1:172.21.15.3:6789/0 conn(0x7f75480adcf0 legacy=0x7f75481644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:42.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754dda6700 1 -- 172.21.15.3:0/2770813586 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75480cd580 con 0x7f7548160880 2023-03-03T06:25:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754dda6700 1 --2- 172.21.15.3:0/2770813586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548160880 0x7f75480c9dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7544009910 tx=0x7f7544009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c859a4e1c962bb2 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754cda4700 1 -- 172.21.15.3:0/2770813586 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7544004230 con 0x7f7548160880 2023-03-03T06:25:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754cda4700 1 -- 172.21.15.3:0/2770813586 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7544004390 con 0x7f7548160880 2023-03-03T06:25:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754cda4700 1 -- 172.21.15.3:0/2770813586 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7544009e60 con 0x7f7548160880 2023-03-03T06:25:42.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754eda8700 1 -- 172.21.15.3:0/2770813586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548160880 msgr2=0x7f75480c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754eda8700 1 --2- 172.21.15.3:0/2770813586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548160880 0x7f75480c9dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7544009910 tx=0x7f7544009c20 comp rx=0 tx=0).stop 2023-03-03T06:25:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754eda8700 1 -- 172.21.15.3:0/2770813586 shutdown_connections 2023-03-03T06:25:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754eda8700 1 --2- 172.21.15.3:0/2770813586 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75480aea70 0x7f75480ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754eda8700 1 --2- 172.21.15.3:0/2770813586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7548160880 0x7f75480c9dc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.023+0000 7f754eda8700 1 -- 172.21.15.3:0/2770813586 >> 172.21.15.3:0/2770813586 conn(0x7f7548070a30 msgr2=0x7f7548070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.027+0000 7f754eda8700 1 -- 172.21.15.3:0/2770813586 shutdown_connections 2023-03-03T06:25:42.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.027+0000 7f754eda8700 1 -- 172.21.15.3:0/2770813586 wait complete. 2023-03-03T06:25:42.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754eda8700 1 Processor -- start 2023-03-03T06:25:42.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754eda8700 1 -- start start 2023-03-03T06:25:42.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754eda8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75480adcf0 0x7f7548154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754eda8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75480aea70 0x7f75480c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754eda8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7548160880 0x7f75480b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754eda8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75480c6d40 con 0x7f75480aea70 2023-03-03T06:25:42.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754eda8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7548155490 con 0x7f75480adcf0 2023-03-03T06:25:42.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754eda8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75480c61c0 con 0x7f7548160880 2023-03-03T06:25:42.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754d5a5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75480aea70 0x7f75480c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754dda6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75480adcf0 0x7f7548154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754dda6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75480adcf0 0x7f7548154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39606/0 (socket says 172.21.15.3:39606) 2023-03-03T06:25:42.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754dda6700 1 -- 172.21.15.3:0/793461116 learned_addr learned my addr 172.21.15.3:0/793461116 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:42.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754e5a7700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7548160880 0x7f75480b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754dda6700 1 -- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7548160880 msgr2=0x7f75480b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754dda6700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7548160880 0x7f75480b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754dda6700 1 -- 172.21.15.3:0/793461116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75480aea70 msgr2=0x7f75480c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754dda6700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75480aea70 0x7f75480c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754dda6700 1 -- 172.21.15.3:0/793461116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7544009570 con 0x7f75480adcf0 2023-03-03T06:25:42.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.031+0000 7f754d5a5700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75480aea70 0x7f75480c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:42.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.035+0000 7f754dda6700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75480adcf0 0x7f7548154f50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7544004000 tx=0x7f7544004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:42.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.035+0000 7f753e7fc700 1 -- 172.21.15.3:0/793461116 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754400bde0 con 0x7f75480adcf0 2023-03-03T06:25:42.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.035+0000 7f753e7fc700 1 -- 172.21.15.3:0/793461116 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75440036c0 con 0x7f75480adcf0 2023-03-03T06:25:42.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.035+0000 7f753e7fc700 1 -- 172.21.15.3:0/793461116 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7544003a50 con 0x7f75480adcf0 2023-03-03T06:25:42.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.035+0000 7f754eda8700 1 -- 172.21.15.3:0/793461116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75480cdb40 con 0x7f75480adcf0 2023-03-03T06:25:42.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.035+0000 7f754eda8700 1 -- 172.21.15.3:0/793461116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75480cd860 con 0x7f75480adcf0 2023-03-03T06:25:42.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.039+0000 7f754eda8700 1 -- 172.21.15.3:0/793461116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75480cde20 con 0x7f75480adcf0 2023-03-03T06:25:42.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.039+0000 7f753e7fc700 1 -- 172.21.15.3:0/793461116 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7544017020 con 0x7f75480adcf0 2023-03-03T06:25:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.043+0000 7f753e7fc700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f752c064f60 0x7f752c067410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.043+0000 7f753e7fc700 1 -- 172.21.15.3:0/793461116 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f75440839c0 con 0x7f75480adcf0 2023-03-03T06:25:42.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.043+0000 7f754d5a5700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f752c064f60 0x7f752c067410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.043+0000 7f754d5a5700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f752c064f60 0x7f752c067410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f75480c8970 tx=0x7f7538009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:42.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.047+0000 7f753e7fc700 1 -- 172.21.15.3:0/793461116 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7544051800 con 0x7f75480adcf0 2023-03-03T06:25:42.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.263+0000 7f754eda8700 1 -- 172.21.15.3:0/793461116 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7548073e90 con 0x7f75480adcf0 2023-03-03T06:25:42.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.263+0000 7f753e7fc700 1 -- 172.21.15.3:0/793461116 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4635 (secure 0 0 0) 0x7f7544054e20 con 0x7f75480adcf0 2023-03-03T06:25:42.268 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:42.269 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2744699972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2744699972},{"type":"v1","addr":"172.21.15.3:6834","nonce":2744699972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/708054272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":708054272},{"type":"v1","addr":"172.21.15.3:6835","nonce":708054272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2935923413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2935923413},{"type":"v1","addr":"172.21.15.32:6837","nonce":2935923413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-03T06:25:13.798672+0000","modified":"2023-03-03T06:25:41.755082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/691220172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":691220172},{"type":"v1","addr":"172.21.15.32:6836","nonce":691220172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-03-03T06:25:42.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.267+0000 7f754eda8700 1 -- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f752c064f60 msgr2=0x7f752c067410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.267+0000 7f754eda8700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f752c064f60 0x7f752c067410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f75480c8970 tx=0x7f7538009110 comp rx=0 tx=0).stop 2023-03-03T06:25:42.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.267+0000 7f754eda8700 1 -- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75480adcf0 msgr2=0x7f7548154f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.267+0000 7f754eda8700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75480adcf0 0x7f7548154f50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7544004000 tx=0x7f7544004770 comp rx=0 tx=0).stop 2023-03-03T06:25:42.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.267+0000 7f754eda8700 1 -- 172.21.15.3:0/793461116 shutdown_connections 2023-03-03T06:25:42.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.267+0000 7f754eda8700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f752c064f60 0x7f752c067410 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.267+0000 7f754eda8700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7548160880 0x7f75480b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.267+0000 7f754eda8700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75480aea70 0x7f75480c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.267+0000 7f754eda8700 1 --2- 172.21.15.3:0/793461116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75480adcf0 0x7f7548154f50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.267+0000 7f754eda8700 1 -- 172.21.15.3:0/793461116 >> 172.21.15.3:0/793461116 conn(0x7f7548070a30 msgr2=0x7f754815e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:42.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.267+0000 7f754eda8700 1 -- 172.21.15.3:0/793461116 shutdown_connections 2023-03-03T06:25:42.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.267+0000 7f754eda8700 1 -- 172.21.15.3:0/793461116 wait complete. 2023-03-03T06:25:42.276 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2023-03-03T06:25:42.308 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:25:42.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.515+0000 7f38e9dcb700 1 Processor -- start 2023-03-03T06:25:42.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.515+0000 7f38e9dcb700 1 -- start start 2023-03-03T06:25:42.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e9dcb700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38e406a780 0x7f38e4158440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e9dcb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38e415c750 0x7f38e40ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e9dcb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e40ca2a0 con 0x7f38e40aee70 2023-03-03T06:25:42.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e9dcb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e40cdd30 con 0x7f38e415c750 2023-03-03T06:25:42.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e9dcb700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e4158980 con 0x7f38e406a780 2023-03-03T06:25:42.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e95ca700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38e415c750 0x7f38e40ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e95ca700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f38e415c750 0x7f38e40ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43252/0 (socket says 172.21.15.3:43252) 2023-03-03T06:25:42.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38dbfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f38e40aee70 0x7f38e415c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33602/0 (socket says 172.21.15.3:33602) 2023-03-03T06:25:42.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e95ca700 1 -- 172.21.15.3:0/3455301893 learned_addr learned my addr 172.21.15.3:0/3455301893 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:42.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e8dc9700 1 --2- 172.21.15.3:0/3455301893 >> v2:172.21.15.32:3300/0 conn(0x7f38e406a780 0x7f38e4158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e95ca700 1 -- 172.21.15.3:0/3455301893 >> v2:172.21.15.32:3300/0 conn(0x7f38e406a780 msgr2=0x7f38e4158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:42.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e95ca700 1 --2- 172.21.15.3:0/3455301893 >> v2:172.21.15.32:3300/0 conn(0x7f38e406a780 0x7f38e4158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e95ca700 1 -- 172.21.15.3:0/3455301893 >> v1:172.21.15.3:6789/0 conn(0x7f38e40aee70 legacy=0x7f38e415c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e95ca700 1 -- 172.21.15.3:0/3455301893 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e40ca4d0 con 0x7f38e415c750 2023-03-03T06:25:42.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38db7fe700 1 -- 172.21.15.3:0/3455301893 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645679948 0 0) 0x7f38e40ca2a0 con 0x7f38e40aee70 2023-03-03T06:25:42.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e95ca700 1 --2- 172.21.15.3:0/3455301893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e415c750 0x7f38e40ce1c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38e40717e0 tx=0x7f38dc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e6a0d744f91adc server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:42.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38db7fe700 1 -- 172.21.15.3:0/3455301893 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00b8a0 con 0x7f38e415c750 2023-03-03T06:25:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38db7fe700 1 -- 172.21.15.3:0/3455301893 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc00ba00 con 0x7f38e415c750 2023-03-03T06:25:42.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38db7fe700 1 -- 172.21.15.3:0/3455301893 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00bd90 con 0x7f38e415c750 2023-03-03T06:25:42.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e9dcb700 1 -- 172.21.15.3:0/3455301893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e415c750 msgr2=0x7f38e40ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e9dcb700 1 --2- 172.21.15.3:0/3455301893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e415c750 0x7f38e40ce1c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38e40717e0 tx=0x7f38dc009bf0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e9dcb700 1 -- 172.21.15.3:0/3455301893 shutdown_connections 2023-03-03T06:25:42.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e9dcb700 1 --2- 172.21.15.3:0/3455301893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e415c750 0x7f38e40ce1c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e9dcb700 1 --2- 172.21.15.3:0/3455301893 >> v2:172.21.15.32:3300/0 conn(0x7f38e406a780 0x7f38e4158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.519+0000 7f38e9dcb700 1 -- 172.21.15.3:0/3455301893 >> 172.21.15.3:0/3455301893 conn(0x7f38e4070a30 msgr2=0x7f38e4070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:42.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.523+0000 7f38e9dcb700 1 -- 172.21.15.3:0/3455301893 shutdown_connections 2023-03-03T06:25:42.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.523+0000 7f38e9dcb700 1 -- 172.21.15.3:0/3455301893 wait complete. 2023-03-03T06:25:42.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.523+0000 7f38e9dcb700 1 Processor -- start 2023-03-03T06:25:42.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.523+0000 7f38e9dcb700 1 -- start start 2023-03-03T06:25:42.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38e9dcb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38e406a780 0x7f38e41578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38e9dcb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e40aee70 0x7f38e40b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38e9dcb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e415c750 0x7f38e41565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38e9dcb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41e2130 con 0x7f38e40aee70 2023-03-03T06:25:42.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38e9dcb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41e1e50 con 0x7f38e415c750 2023-03-03T06:25:42.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38e9dcb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40b3130 con 0x7f38e406a780 2023-03-03T06:25:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38dbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e40aee70 0x7f38e40b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38dbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e40aee70 0x7f38e40b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43254/0 (socket says 172.21.15.3:43254) 2023-03-03T06:25:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38dbfff700 1 -- 172.21.15.3:0/773839336 learned_addr learned my addr 172.21.15.3:0/773839336 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38e95ca700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e415c750 0x7f38e41565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38e8dc9700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38e406a780 0x7f38e41578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38dbfff700 1 -- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38e406a780 msgr2=0x7f38e41578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38dbfff700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38e406a780 0x7f38e41578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38dbfff700 1 -- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e415c750 msgr2=0x7f38e41565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38dbfff700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e415c750 0x7f38e41565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38dbfff700 1 -- 172.21.15.3:0/773839336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc009570 con 0x7f38e40aee70 2023-03-03T06:25:42.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38e8dc9700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38e406a780 0x7f38e41578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:42.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38dbfff700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e40aee70 0x7f38e40b3c10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38d400c750 tx=0x7f38d400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38d97fa700 1 -- 172.21.15.3:0/773839336 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4004ca0 con 0x7f38e40aee70 2023-03-03T06:25:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38d97fa700 1 -- 172.21.15.3:0/773839336 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d4004e00 con 0x7f38e40aee70 2023-03-03T06:25:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38d97fa700 1 -- 172.21.15.3:0/773839336 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4007310 con 0x7f38e40aee70 2023-03-03T06:25:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38e9dcb700 1 -- 172.21.15.3:0/773839336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e40cd9c0 con 0x7f38e40aee70 2023-03-03T06:25:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.527+0000 7f38e9dcb700 1 -- 172.21.15.3:0/773839336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e40cd6e0 con 0x7f38e40aee70 2023-03-03T06:25:42.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.531+0000 7f38e9dcb700 1 -- 172.21.15.3:0/773839336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e40ca7b0 con 0x7f38e40aee70 2023-03-03T06:25:42.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.531+0000 7f38d97fa700 1 -- 172.21.15.3:0/773839336 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f38d401a020 con 0x7f38e40aee70 2023-03-03T06:25:42.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.535+0000 7f38d97fa700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f38d0064ee0 0x7f38d0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.535+0000 7f38d97fa700 1 -- 172.21.15.3:0/773839336 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f38d4081ea0 con 0x7f38e40aee70 2023-03-03T06:25:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.535+0000 7f38e8dc9700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f38d0064ee0 0x7f38d0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.539+0000 7f38e8dc9700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f38d0064ee0 0x7f38d0067390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009430 tx=0x7f38e000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:42.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.539+0000 7f38d97fa700 1 -- 172.21.15.3:0/773839336 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f38d404fc60 con 0x7f38e40aee70 2023-03-03T06:25:42.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.715+0000 7f38e9dcb700 1 -- 172.21.15.3:0/773839336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38e4073e90 con 0x7f38e40aee70 2023-03-03T06:25:42.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.719+0000 7f38d97fa700 1 -- 172.21.15.3:0/773839336 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19556 (secure 0 0 0) 0x7f38d4053280 con 0x7f38e40aee70 2023-03-03T06:25:42.723 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:42.724 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:18.863350+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:25:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.723+0000 7f38e9dcb700 1 -- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f38d0064ee0 msgr2=0x7f38d0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.723+0000 7f38e9dcb700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f38d0064ee0 0x7f38d0067390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009430 tx=0x7f38e000b040 comp rx=0 tx=0).stop 2023-03-03T06:25:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.723+0000 7f38e9dcb700 1 -- 172.21.15.3:0/773839336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e40aee70 msgr2=0x7f38e40b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.723+0000 7f38e9dcb700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e40aee70 0x7f38e40b3c10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38d400c750 tx=0x7f38d400ca60 comp rx=0 tx=0).stop 2023-03-03T06:25:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.727+0000 7f38e9dcb700 1 -- 172.21.15.3:0/773839336 shutdown_connections 2023-03-03T06:25:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.727+0000 7f38e9dcb700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f38d0064ee0 0x7f38d0067390 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.727+0000 7f38e9dcb700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38e415c750 0x7f38e41565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.727+0000 7f38e9dcb700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38e40aee70 0x7f38e40b3c10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.727+0000 7f38e9dcb700 1 --2- 172.21.15.3:0/773839336 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f38e406a780 0x7f38e41578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.727+0000 7f38e9dcb700 1 -- 172.21.15.3:0/773839336 >> 172.21.15.3:0/773839336 conn(0x7f38e4070a30 msgr2=0x7f38e4154060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.727+0000 7f38e9dcb700 1 -- 172.21.15.3:0/773839336 shutdown_connections 2023-03-03T06:25:42.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.727+0000 7f38e9dcb700 1 -- 172.21.15.3:0/773839336 wait complete. 2023-03-03T06:25:42.765 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-03-03T06:25:42.766 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-03T06:25:42.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.967+0000 7f0234b64700 1 Processor -- start 2023-03-03T06:25:42.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.967+0000 7f0234b64700 1 -- start start 2023-03-03T06:25:42.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f0234b64700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f02300c9fc0 0x7f023015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f0234b64700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02300cb770 0x7f02300cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f0234b64700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023015d940 con 0x7f023015c3d0 2023-03-03T06:25:42.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f0234b64700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023015cdc0 con 0x7f02300c9fc0 2023-03-03T06:25:42.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f0234b64700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023015d0a0 con 0x7f02300cb770 2023-03-03T06:25:42.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f0226ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f02300c9fc0 0x7f023015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f0226ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f02300c9fc0 0x7f023015c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43266/0 (socket says 172.21.15.3:43266) 2023-03-03T06:25:42.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f0226ffd700 1 -- 172.21.15.3:0/3164293084 learned_addr learned my addr 172.21.15.3:0/3164293084 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:42.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f022f7fe700 1 --1- 172.21.15.3:0/3164293084 >> v1:172.21.15.3:6789/0 conn(0x7f023015c3d0 0x7f02300c98b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33616/0 (socket says 172.21.15.3:33616) 2023-03-03T06:25:42.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f022ffff700 1 --2- 172.21.15.3:0/3164293084 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02300cb770 0x7f02300cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f0226ffd700 1 -- 172.21.15.3:0/3164293084 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02300cb770 msgr2=0x7f02300cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:42.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f0226ffd700 1 --2- 172.21.15.3:0/3164293084 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02300cb770 0x7f02300cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f0226ffd700 1 -- 172.21.15.3:0/3164293084 >> v1:172.21.15.3:6789/0 conn(0x7f023015c3d0 legacy=0x7f02300c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f0226ffd700 1 -- 172.21.15.3:0/3164293084 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023015d380 con 0x7f02300c9fc0 2023-03-03T06:25:42.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f0226ffd700 1 --2- 172.21.15.3:0/3164293084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02300c9fc0 0x7f023015c7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0218009e50 tx=0x7f021800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22ae3019292ba97 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:42.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.971+0000 7f022effd700 1 -- 172.21.15.3:0/3164293084 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021800e070 con 0x7f02300c9fc0 2023-03-03T06:25:42.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.975+0000 7f022effd700 1 -- 172.21.15.3:0/3164293084 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021800b9c0 con 0x7f02300c9fc0 2023-03-03T06:25:42.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.975+0000 7f022effd700 1 -- 172.21.15.3:0/3164293084 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021800bd70 con 0x7f02300c9fc0 2023-03-03T06:25:42.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.975+0000 7f0234b64700 1 -- 172.21.15.3:0/3164293084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02300c9fc0 msgr2=0x7f023015c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.975+0000 7f0234b64700 1 --2- 172.21.15.3:0/3164293084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02300c9fc0 0x7f023015c7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0218009e50 tx=0x7f021800b330 comp rx=0 tx=0).stop 2023-03-03T06:25:42.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.975+0000 7f0234b64700 1 -- 172.21.15.3:0/3164293084 shutdown_connections 2023-03-03T06:25:42.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.975+0000 7f0234b64700 1 --2- 172.21.15.3:0/3164293084 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02300cb770 0x7f02300cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.975+0000 7f0234b64700 1 --2- 172.21.15.3:0/3164293084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02300c9fc0 0x7f023015c7a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.975+0000 7f0234b64700 1 -- 172.21.15.3:0/3164293084 >> 172.21.15.3:0/3164293084 conn(0x7f0230070a30 msgr2=0x7f0230070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:42.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.975+0000 7f0234b64700 1 -- 172.21.15.3:0/3164293084 shutdown_connections 2023-03-03T06:25:42.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.975+0000 7f0234b64700 1 -- 172.21.15.3:0/3164293084 wait complete. 2023-03-03T06:25:42.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.979+0000 7f0234b64700 1 Processor -- start 2023-03-03T06:25:42.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.979+0000 7f0234b64700 1 -- start start 2023-03-03T06:25:42.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.979+0000 7f0234b64700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02300c9fc0 0x7f023014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.979+0000 7f0234b64700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02300cb770 0x7f023014f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.979+0000 7f0234b64700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f023015c3d0 0x7f023014fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.979+0000 7f0234b64700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02300c6520 con 0x7f023015c3d0 2023-03-03T06:25:42.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.979+0000 7f0234b64700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02300c5910 con 0x7f02300cb770 2023-03-03T06:25:42.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.979+0000 7f0234b64700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02300c5bf0 con 0x7f02300c9fc0 2023-03-03T06:25:42.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.979+0000 7f022ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f023015c3d0 0x7f023014fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.979+0000 7f022ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f023015c3d0 0x7f023014fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43280/0 (socket says 172.21.15.3:43280) 2023-03-03T06:25:42.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.979+0000 7f022ffff700 1 -- 172.21.15.3:0/4125864794 learned_addr learned my addr 172.21.15.3:0/4125864794 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:42.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.979+0000 7f022f7fe700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02300c9fc0 0x7f023014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.979+0000 7f0226ffd700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02300cb770 0x7f023014f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.983+0000 7f022ffff700 1 -- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02300c9fc0 msgr2=0x7f023014eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.983+0000 7f022ffff700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02300c9fc0 0x7f023014eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.983+0000 7f022ffff700 1 -- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02300cb770 msgr2=0x7f023014f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:42.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.983+0000 7f022ffff700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02300cb770 0x7f023014f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:42.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.983+0000 7f022ffff700 1 -- 172.21.15.3:0/4125864794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0218009b30 con 0x7f023015c3d0 2023-03-03T06:25:42.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.983+0000 7f022f7fe700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02300c9fc0 0x7f023014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:42.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.983+0000 7f022ffff700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f023015c3d0 0x7f023014fe80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f022800e810 tx=0x7f022800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:42.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.983+0000 7f022cff9700 1 -- 172.21.15.3:0/4125864794 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0228013610 con 0x7f023015c3d0 2023-03-03T06:25:42.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.983+0000 7f022cff9700 1 -- 172.21.15.3:0/4125864794 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0228013770 con 0x7f023015c3d0 2023-03-03T06:25:42.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.983+0000 7f022cff9700 1 -- 172.21.15.3:0/4125864794 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0228013b20 con 0x7f023015c3d0 2023-03-03T06:25:42.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.983+0000 7f0234b64700 1 -- 172.21.15.3:0/4125864794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02300cbc10 con 0x7f023015c3d0 2023-03-03T06:25:42.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.983+0000 7f0234b64700 1 -- 172.21.15.3:0/4125864794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023015d660 con 0x7f023015c3d0 2023-03-03T06:25:42.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.983+0000 7f0234b64700 1 -- 172.21.15.3:0/4125864794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02300cb150 con 0x7f023015c3d0 2023-03-03T06:25:42.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.987+0000 7f022cff9700 1 -- 172.21.15.3:0/4125864794 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f0228023070 con 0x7f023015c3d0 2023-03-03T06:25:42.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.991+0000 7f022cff9700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0210064f30 0x7f02100673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:42.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.991+0000 7f022cff9700 1 -- 172.21.15.3:0/4125864794 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0228081e40 con 0x7f023015c3d0 2023-03-03T06:25:42.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.991+0000 7f022f7fe700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0210064f30 0x7f02100673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:42.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.991+0000 7f022f7fe700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0210064f30 0x7f02100673e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0220000f60 tx=0x7f0220007df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:42.999+0000 7f022cff9700 1 -- 172.21.15.3:0/4125864794 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f022804fc00 con 0x7f023015c3d0 2023-03-03T06:25:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.171+0000 7f0234b64700 1 -- 172.21.15.3:0/4125864794 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0230073e90 con 0x7f0210064f30 2023-03-03T06:25:43.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.195+0000 7f022cff9700 1 -- 172.21.15.3:0/4125864794 <== mgr.4105 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367071 (secure 0 0 0) 0x7f0230073e90 con 0x7f0210064f30 2023-03-03T06:25:43.200 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.203+0000 7f0234b64700 1 -- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0210064f30 msgr2=0x7f02100673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.203+0000 7f0234b64700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0210064f30 0x7f02100673e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0220000f60 tx=0x7f0220007df0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.203+0000 7f0234b64700 1 -- 172.21.15.3:0/4125864794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f023015c3d0 msgr2=0x7f023014fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:43.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.203+0000 7f0234b64700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f023015c3d0 0x7f023014fe80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f022800e810 tx=0x7f022800eb20 comp rx=0 tx=0).stop 2023-03-03T06:25:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.203+0000 7f0234b64700 1 -- 172.21.15.3:0/4125864794 shutdown_connections 2023-03-03T06:25:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.203+0000 7f0234b64700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0210064f30 0x7f02100673e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.203+0000 7f0234b64700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f023015c3d0 0x7f023014fe80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.203+0000 7f0234b64700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02300cb770 0x7f023014f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.203+0000 7f0234b64700 1 --2- 172.21.15.3:0/4125864794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02300c9fc0 0x7f023014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.203+0000 7f0234b64700 1 -- 172.21.15.3:0/4125864794 >> 172.21.15.3:0/4125864794 conn(0x7f0230070a30 msgr2=0x7f023006e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:43.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.203+0000 7f0234b64700 1 -- 172.21.15.3:0/4125864794 shutdown_connections 2023-03-03T06:25:43.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.203+0000 7f0234b64700 1 -- 172.21.15.3:0/4125864794 wait complete. 2023-03-03T06:25:43.211 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-03-03T06:25:43.243 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2023-03-03T06:25:42.408169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594895,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":84232,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":80576,"kb_avail":754890488,"statfs":{"total":773094113280,"available":773007859712,"internally_reserved":0,"allocated":3678208,"data_stored":2186366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":82509824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005964"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461806+0000","last_change":"2023-03-03T06:25:18.375133+0000","last_active":"2023-03-03T06:25:19.461806+0000","last_peered":"2023-03-03T06:25:19.461806+0000","last_clean":"2023-03-03T06:25:19.461806+0000","last_became_active":"2023-03-03T06:25:18.374500+0000","last_became_peered":"2023-03-03T06:25:18.374500+0000","last_unstale":"2023-03-03T06:25:19.461806+0000","last_undegraded":"2023-03-03T06:25:19.461806+0000","last_fullsized":"2023-03-03T06:25:19.461806+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:30.336080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343444+0000","last_change":"2023-03-03T06:25:13.301166+0000","last_active":"2023-03-03T06:25:18.343444+0000","last_peered":"2023-03-03T06:25:18.343444+0000","last_clean":"2023-03-03T06:25:18.343444+0000","last_became_active":"2023-03-03T06:25:13.300749+0000","last_became_peered":"2023-03-03T06:25:13.300749+0000","last_unstale":"2023-03-03T06:25:18.343444+0000","last_undegraded":"2023-03-03T06:25:18.343444+0000","last_fullsized":"2023-03-03T06:25:18.343444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335467+0000","last_change":"2023-03-03T06:25:13.839287+0000","last_active":"2023-03-03T06:25:18.335467+0000","last_peered":"2023-03-03T06:25:18.335467+0000","last_clean":"2023-03-03T06:25:18.335467+0000","last_became_active":"2023-03-03T06:25:13.838935+0000","last_became_peered":"2023-03-03T06:25:13.838935+0000","last_unstale":"2023-03-03T06:25:18.335467+0000","last_undegraded":"2023-03-03T06:25:18.335467+0000","last_fullsized":"2023-03-03T06:25:18.335467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:48:01.152785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.876352+0000","last_change":"2023-03-03T06:25:18.362828+0000","last_active":"2023-03-03T06:25:18.876352+0000","last_peered":"2023-03-03T06:25:18.876352+0000","last_clean":"2023-03-03T06:25:18.876352+0000","last_became_active":"2023-03-03T06:25:18.362423+0000","last_became_peered":"2023-03-03T06:25:18.362423+0000","last_unstale":"2023-03-03T06:25:18.876352+0000","last_undegraded":"2023-03-03T06:25:18.876352+0000","last_fullsized":"2023-03-03T06:25:18.876352+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:04:44.947509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.339522+0000","last_change":"2023-03-03T06:25:13.307540+0000","last_active":"2023-03-03T06:25:18.339522+0000","last_peered":"2023-03-03T06:25:18.339522+0000","last_clean":"2023-03-03T06:25:18.339522+0000","last_became_active":"2023-03-03T06:25:13.307229+0000","last_became_peered":"2023-03-03T06:25:13.307229+0000","last_unstale":"2023-03-03T06:25:18.339522+0000","last_undegraded":"2023-03-03T06:25:18.339522+0000","last_fullsized":"2023-03-03T06:25:18.339522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:09:34.703888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337997+0000","last_change":"2023-03-03T06:25:13.832118+0000","last_active":"2023-03-03T06:25:18.337997+0000","last_peered":"2023-03-03T06:25:18.337997+0000","last_clean":"2023-03-03T06:25:18.337997+0000","last_became_active":"2023-03-03T06:25:13.831744+0000","last_became_peered":"2023-03-03T06:25:13.831744+0000","last_unstale":"2023-03-03T06:25:18.337997+0000","last_undegraded":"2023-03-03T06:25:18.337997+0000","last_fullsized":"2023-03-03T06:25:18.337997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.878930+0000","last_change":"2023-03-03T06:25:18.367879+0000","last_active":"2023-03-03T06:25:18.878930+0000","last_peered":"2023-03-03T06:25:18.878930+0000","last_clean":"2023-03-03T06:25:18.878930+0000","last_became_active":"2023-03-03T06:25:18.367232+0000","last_became_peered":"2023-03-03T06:25:18.367232+0000","last_unstale":"2023-03-03T06:25:18.878930+0000","last_undegraded":"2023-03-03T06:25:18.878930+0000","last_fullsized":"2023-03-03T06:25:18.878930+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:43:20.174095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352877+0000","last_change":"2023-03-03T06:25:13.301087+0000","last_active":"2023-03-03T06:25:18.352877+0000","last_peered":"2023-03-03T06:25:18.352877+0000","last_clean":"2023-03-03T06:25:18.352877+0000","last_became_active":"2023-03-03T06:25:13.300837+0000","last_became_peered":"2023-03-03T06:25:13.300837+0000","last_unstale":"2023-03-03T06:25:18.352877+0000","last_undegraded":"2023-03-03T06:25:18.352877+0000","last_fullsized":"2023-03-03T06:25:18.352877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334901+0000","last_change":"2023-03-03T06:25:13.830954+0000","last_active":"2023-03-03T06:25:18.334901+0000","last_peered":"2023-03-03T06:25:18.334901+0000","last_clean":"2023-03-03T06:25:18.334901+0000","last_became_active":"2023-03-03T06:25:13.830614+0000","last_became_peered":"2023-03-03T06:25:13.830614+0000","last_unstale":"2023-03-03T06:25:18.334901+0000","last_undegraded":"2023-03-03T06:25:18.334901+0000","last_fullsized":"2023-03-03T06:25:18.334901+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.463051+0000","last_change":"2023-03-03T06:25:18.366781+0000","last_active":"2023-03-03T06:25:19.463051+0000","last_peered":"2023-03-03T06:25:19.463051+0000","last_clean":"2023-03-03T06:25:19.463051+0000","last_became_active":"2023-03-03T06:25:18.366175+0000","last_became_peered":"2023-03-03T06:25:18.366175+0000","last_unstale":"2023-03-03T06:25:19.463051+0000","last_undegraded":"2023-03-03T06:25:19.463051+0000","last_fullsized":"2023-03-03T06:25:19.463051+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:30.336080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352547+0000","last_change":"2023-03-03T06:25:13.300495+0000","last_active":"2023-03-03T06:25:18.352547+0000","last_peered":"2023-03-03T06:25:18.352547+0000","last_clean":"2023-03-03T06:25:18.352547+0000","last_became_active":"2023-03-03T06:25:13.300180+0000","last_became_peered":"2023-03-03T06:25:13.300180+0000","last_unstale":"2023-03-03T06:25:18.352547+0000","last_undegraded":"2023-03-03T06:25:18.352547+0000","last_fullsized":"2023-03-03T06:25:18.352547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335053+0000","last_change":"2023-03-03T06:25:13.830988+0000","last_active":"2023-03-03T06:25:18.335053+0000","last_peered":"2023-03-03T06:25:18.335053+0000","last_clean":"2023-03-03T06:25:18.335053+0000","last_became_active":"2023-03-03T06:25:13.826124+0000","last_became_peered":"2023-03-03T06:25:13.826124+0000","last_unstale":"2023-03-03T06:25:18.335053+0000","last_undegraded":"2023-03-03T06:25:18.335053+0000","last_fullsized":"2023-03-03T06:25:18.335053+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461936+0000","last_change":"2023-03-03T06:25:18.372519+0000","last_active":"2023-03-03T06:25:19.461936+0000","last_peered":"2023-03-03T06:25:19.461936+0000","last_clean":"2023-03-03T06:25:19.461936+0000","last_became_active":"2023-03-03T06:25:18.372087+0000","last_became_peered":"2023-03-03T06:25:18.372087+0000","last_unstale":"2023-03-03T06:25:19.461936+0000","last_undegraded":"2023-03-03T06:25:19.461936+0000","last_fullsized":"2023-03-03T06:25:19.461936+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:04.336725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344076+0000","last_change":"2023-03-03T06:25:13.308494+0000","last_active":"2023-03-03T06:25:18.344076+0000","last_peered":"2023-03-03T06:25:18.344076+0000","last_clean":"2023-03-03T06:25:18.344076+0000","last_became_active":"2023-03-03T06:25:13.308110+0000","last_became_peered":"2023-03-03T06:25:13.308110+0000","last_unstale":"2023-03-03T06:25:18.344076+0000","last_undegraded":"2023-03-03T06:25:18.344076+0000","last_fullsized":"2023-03-03T06:25:18.344076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.336445+0000","last_change":"2023-03-03T06:25:13.833412+0000","last_active":"2023-03-03T06:25:18.336445+0000","last_peered":"2023-03-03T06:25:18.336445+0000","last_clean":"2023-03-03T06:25:18.336445+0000","last_became_active":"2023-03-03T06:25:13.833138+0000","last_became_peered":"2023-03-03T06:25:13.833138+0000","last_unstale":"2023-03-03T06:25:18.336445+0000","last_undegraded":"2023-03-03T06:25:18.336445+0000","last_fullsized":"2023-03-03T06:25:18.336445+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:16:17.602774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.876849+0000","last_change":"2023-03-03T06:25:18.354570+0000","last_active":"2023-03-03T06:25:18.876849+0000","last_peered":"2023-03-03T06:25:18.876849+0000","last_clean":"2023-03-03T06:25:18.876849+0000","last_became_active":"2023-03-03T06:25:18.354128+0000","last_became_peered":"2023-03-03T06:25:18.354128+0000","last_unstale":"2023-03-03T06:25:18.876849+0000","last_undegraded":"2023-03-03T06:25:18.876849+0000","last_fullsized":"2023-03-03T06:25:18.876849+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:43:00.515957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335127+0000","last_change":"2023-03-03T06:25:13.307789+0000","last_active":"2023-03-03T06:25:18.335127+0000","last_peered":"2023-03-03T06:25:18.335127+0000","last_clean":"2023-03-03T06:25:18.335127+0000","last_became_active":"2023-03-03T06:25:13.307478+0000","last_became_peered":"2023-03-03T06:25:13.307478+0000","last_unstale":"2023-03-03T06:25:18.335127+0000","last_undegraded":"2023-03-03T06:25:18.335127+0000","last_fullsized":"2023-03-03T06:25:18.335127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:11:32.033289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.339285+0000","last_change":"2023-03-03T06:25:13.830797+0000","last_active":"2023-03-03T06:25:18.339285+0000","last_peered":"2023-03-03T06:25:18.339285+0000","last_clean":"2023-03-03T06:25:18.339285+0000","last_became_active":"2023-03-03T06:25:13.829482+0000","last_became_peered":"2023-03-03T06:25:13.829482+0000","last_unstale":"2023-03-03T06:25:18.339285+0000","last_undegraded":"2023-03-03T06:25:18.339285+0000","last_fullsized":"2023-03-03T06:25:18.339285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:48:01.152785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.459250+0000","last_change":"2023-03-03T06:25:18.358329+0000","last_active":"2023-03-03T06:25:19.459250+0000","last_peered":"2023-03-03T06:25:19.459250+0000","last_clean":"2023-03-03T06:25:19.459250+0000","last_became_active":"2023-03-03T06:25:18.357641+0000","last_became_peered":"2023-03-03T06:25:18.357641+0000","last_unstale":"2023-03-03T06:25:19.459250+0000","last_undegraded":"2023-03-03T06:25:19.459250+0000","last_fullsized":"2023-03-03T06:25:19.459250+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:59.071940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.359353+0000","last_change":"2023-03-03T06:25:13.312665+0000","last_active":"2023-03-03T06:25:18.359353+0000","last_peered":"2023-03-03T06:25:18.359353+0000","last_clean":"2023-03-03T06:25:18.359353+0000","last_became_active":"2023-03-03T06:25:13.312255+0000","last_became_peered":"2023-03-03T06:25:13.312255+0000","last_unstale":"2023-03-03T06:25:18.359353+0000","last_undegraded":"2023-03-03T06:25:18.359353+0000","last_fullsized":"2023-03-03T06:25:18.359353+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344419+0000","last_change":"2023-03-03T06:25:13.847886+0000","last_active":"2023-03-03T06:25:18.344419+0000","last_peered":"2023-03-03T06:25:18.344419+0000","last_clean":"2023-03-03T06:25:18.344419+0000","last_became_active":"2023-03-03T06:25:13.847620+0000","last_became_peered":"2023-03-03T06:25:13.847620+0000","last_unstale":"2023-03-03T06:25:18.344419+0000","last_undegraded":"2023-03-03T06:25:18.344419+0000","last_fullsized":"2023-03-03T06:25:18.344419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:48:01.152785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.875799+0000","last_change":"2023-03-03T06:25:18.374567+0000","last_active":"2023-03-03T06:25:18.875799+0000","last_peered":"2023-03-03T06:25:18.875799+0000","last_clean":"2023-03-03T06:25:18.875799+0000","last_became_active":"2023-03-03T06:25:18.374172+0000","last_became_peered":"2023-03-03T06:25:18.374172+0000","last_unstale":"2023-03-03T06:25:18.875799+0000","last_undegraded":"2023-03-03T06:25:18.875799+0000","last_fullsized":"2023-03-03T06:25:18.875799+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:11:32.109566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341581+0000","last_change":"2023-03-03T06:25:13.310405+0000","last_active":"2023-03-03T06:25:18.341581+0000","last_peered":"2023-03-03T06:25:18.341581+0000","last_clean":"2023-03-03T06:25:18.341581+0000","last_became_active":"2023-03-03T06:25:13.310029+0000","last_became_peered":"2023-03-03T06:25:13.310029+0000","last_unstale":"2023-03-03T06:25:18.341581+0000","last_undegraded":"2023-03-03T06:25:18.341581+0000","last_fullsized":"2023-03-03T06:25:18.341581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:16:16.586666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343606+0000","last_change":"2023-03-03T06:25:13.839392+0000","last_active":"2023-03-03T06:25:18.343606+0000","last_peered":"2023-03-03T06:25:18.343606+0000","last_clean":"2023-03-03T06:25:18.343606+0000","last_became_active":"2023-03-03T06:25:13.839078+0000","last_became_peered":"2023-03-03T06:25:13.839078+0000","last_unstale":"2023-03-03T06:25:18.343606+0000","last_undegraded":"2023-03-03T06:25:18.343606+0000","last_fullsized":"2023-03-03T06:25:18.343606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:42:30.992559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341473+0000","last_change":"2023-03-03T06:25:13.836064+0000","last_active":"2023-03-03T06:25:18.341473+0000","last_peered":"2023-03-03T06:25:18.341473+0000","last_clean":"2023-03-03T06:25:18.341473+0000","last_became_active":"2023-03-03T06:25:13.835743+0000","last_became_peered":"2023-03-03T06:25:13.835743+0000","last_unstale":"2023-03-03T06:25:18.341473+0000","last_undegraded":"2023-03-03T06:25:18.341473+0000","last_fullsized":"2023-03-03T06:25:18.341473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:52:13.551129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343523+0000","last_change":"2023-03-03T06:25:13.293588+0000","last_active":"2023-03-03T06:25:18.343523+0000","last_peered":"2023-03-03T06:25:18.343523+0000","last_clean":"2023-03-03T06:25:18.343523+0000","last_became_active":"2023-03-03T06:25:13.292998+0000","last_became_peered":"2023-03-03T06:25:13.292998+0000","last_unstale":"2023-03-03T06:25:18.343523+0000","last_undegraded":"2023-03-03T06:25:18.343523+0000","last_fullsized":"2023-03-03T06:25:18.343523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:17:34.388274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.459022+0000","last_change":"2023-03-03T06:25:18.365923+0000","last_active":"2023-03-03T06:25:19.459022+0000","last_peered":"2023-03-03T06:25:19.459022+0000","last_clean":"2023-03-03T06:25:19.459022+0000","last_became_active":"2023-03-03T06:25:18.365397+0000","last_became_peered":"2023-03-03T06:25:18.365397+0000","last_unstale":"2023-03-03T06:25:19.459022+0000","last_undegraded":"2023-03-03T06:25:19.459022+0000","last_fullsized":"2023-03-03T06:25:19.459022+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:45:30.096214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337824+0000","last_change":"2023-03-03T06:25:13.837989+0000","last_active":"2023-03-03T06:25:18.337824+0000","last_peered":"2023-03-03T06:25:18.337824+0000","last_clean":"2023-03-03T06:25:18.337824+0000","last_became_active":"2023-03-03T06:25:13.837724+0000","last_became_peered":"2023-03-03T06:25:13.837724+0000","last_unstale":"2023-03-03T06:25:18.337824+0000","last_undegraded":"2023-03-03T06:25:18.337824+0000","last_fullsized":"2023-03-03T06:25:18.337824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353186+0000","last_change":"2023-03-03T06:25:13.302153+0000","last_active":"2023-03-03T06:25:18.353186+0000","last_peered":"2023-03-03T06:25:18.353186+0000","last_clean":"2023-03-03T06:25:18.353186+0000","last_became_active":"2023-03-03T06:25:13.301751+0000","last_became_peered":"2023-03-03T06:25:13.301751+0000","last_unstale":"2023-03-03T06:25:18.353186+0000","last_undegraded":"2023-03-03T06:25:18.353186+0000","last_fullsized":"2023-03-03T06:25:18.353186+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.462897+0000","last_change":"2023-03-03T06:25:18.366010+0000","last_active":"2023-03-03T06:25:19.462897+0000","last_peered":"2023-03-03T06:25:19.462897+0000","last_clean":"2023-03-03T06:25:19.462897+0000","last_became_active":"2023-03-03T06:25:18.362457+0000","last_became_peered":"2023-03-03T06:25:18.362457+0000","last_unstale":"2023-03-03T06:25:19.462897+0000","last_undegraded":"2023-03-03T06:25:19.462897+0000","last_fullsized":"2023-03-03T06:25:19.462897+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:17:02.442524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350698+0000","last_change":"2023-03-03T06:25:13.312520+0000","last_active":"2023-03-03T06:25:18.350698+0000","last_peered":"2023-03-03T06:25:18.350698+0000","last_clean":"2023-03-03T06:25:18.350698+0000","last_became_active":"2023-03-03T06:25:13.312105+0000","last_became_peered":"2023-03-03T06:25:13.312105+0000","last_unstale":"2023-03-03T06:25:18.350698+0000","last_undegraded":"2023-03-03T06:25:18.350698+0000","last_fullsized":"2023-03-03T06:25:18.350698+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:51:29.967970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358702+0000","last_change":"2023-03-03T06:25:13.835492+0000","last_active":"2023-03-03T06:25:18.358702+0000","last_peered":"2023-03-03T06:25:18.358702+0000","last_clean":"2023-03-03T06:25:18.358702+0000","last_became_active":"2023-03-03T06:25:13.835173+0000","last_became_peered":"2023-03-03T06:25:13.835173+0000","last_unstale":"2023-03-03T06:25:18.358702+0000","last_undegraded":"2023-03-03T06:25:18.358702+0000","last_fullsized":"2023-03-03T06:25:18.358702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467726+0000","last_change":"2023-03-03T06:25:18.362177+0000","last_active":"2023-03-03T06:25:19.467726+0000","last_peered":"2023-03-03T06:25:19.467726+0000","last_clean":"2023-03-03T06:25:19.467726+0000","last_became_active":"2023-03-03T06:25:18.361833+0000","last_became_peered":"2023-03-03T06:25:18.361833+0000","last_unstale":"2023-03-03T06:25:19.467726+0000","last_undegraded":"2023-03-03T06:25:19.467726+0000","last_fullsized":"2023-03-03T06:25:19.467726+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:04:44.947509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:32.598145+0000","last_change":"2023-03-03T06:25:13.302558+0000","last_active":"2023-03-03T06:25:32.598145+0000","last_peered":"2023-03-03T06:25:32.598145+0000","last_clean":"2023-03-03T06:25:32.598145+0000","last_became_active":"2023-03-03T06:25:13.301983+0000","last_became_peered":"2023-03-03T06:25:13.301983+0000","last_unstale":"2023-03-03T06:25:32.598145+0000","last_undegraded":"2023-03-03T06:25:32.598145+0000","last_fullsized":"2023-03-03T06:25:32.598145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:34:41.158814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3745,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334685+0000","last_change":"2023-03-03T06:25:13.835359+0000","last_active":"2023-03-03T06:25:18.334685+0000","last_peered":"2023-03-03T06:25:18.334685+0000","last_clean":"2023-03-03T06:25:18.334685+0000","last_became_active":"2023-03-03T06:25:13.835103+0000","last_became_peered":"2023-03-03T06:25:13.835103+0000","last_unstale":"2023-03-03T06:25:18.334685+0000","last_undegraded":"2023-03-03T06:25:18.334685+0000","last_fullsized":"2023-03-03T06:25:18.334685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461833+0000","last_change":"2023-03-03T06:25:18.376174+0000","last_active":"2023-03-03T06:25:19.461833+0000","last_peered":"2023-03-03T06:25:19.461833+0000","last_clean":"2023-03-03T06:25:19.461833+0000","last_became_active":"2023-03-03T06:25:18.375779+0000","last_became_peered":"2023-03-03T06:25:18.375779+0000","last_unstale":"2023-03-03T06:25:19.461833+0000","last_undegraded":"2023-03-03T06:25:19.461833+0000","last_fullsized":"2023-03-03T06:25:19.461833+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:40:49.542836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335280+0000","last_change":"2023-03-03T06:25:13.837277+0000","last_active":"2023-03-03T06:25:18.335280+0000","last_peered":"2023-03-03T06:25:18.335280+0000","last_clean":"2023-03-03T06:25:18.335280+0000","last_became_active":"2023-03-03T06:25:13.836510+0000","last_became_peered":"2023-03-03T06:25:13.836510+0000","last_unstale":"2023-03-03T06:25:18.335280+0000","last_undegraded":"2023-03-03T06:25:18.335280+0000","last_fullsized":"2023-03-03T06:25:18.335280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343682+0000","last_change":"2023-03-03T06:25:07.205700+0000","last_active":"2023-03-03T06:25:18.343682+0000","last_peered":"2023-03-03T06:25:18.343682+0000","last_clean":"2023-03-03T06:25:18.343682+0000","last_became_active":"2023-03-03T06:25:07.205269+0000","last_became_peered":"2023-03-03T06:25:07.205269+0000","last_unstale":"2023-03-03T06:25:18.343682+0000","last_undegraded":"2023-03-03T06:25:18.343682+0000","last_fullsized":"2023-03-03T06:25:18.343682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:17:53.442570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344712+0000","last_change":"2023-03-03T06:25:13.309537+0000","last_active":"2023-03-03T06:25:18.344712+0000","last_peered":"2023-03-03T06:25:18.344712+0000","last_clean":"2023-03-03T06:25:18.344712+0000","last_became_active":"2023-03-03T06:25:13.309208+0000","last_became_peered":"2023-03-03T06:25:13.309208+0000","last_unstale":"2023-03-03T06:25:18.344712+0000","last_undegraded":"2023-03-03T06:25:18.344712+0000","last_fullsized":"2023-03-03T06:25:18.344712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:11:45.788884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467622+0000","last_change":"2023-03-03T06:25:18.367070+0000","last_active":"2023-03-03T06:25:19.467622+0000","last_peered":"2023-03-03T06:25:19.467622+0000","last_clean":"2023-03-03T06:25:19.467622+0000","last_became_active":"2023-03-03T06:25:18.366674+0000","last_became_peered":"2023-03-03T06:25:18.366674+0000","last_unstale":"2023-03-03T06:25:19.467622+0000","last_undegraded":"2023-03-03T06:25:19.467622+0000","last_fullsized":"2023-03-03T06:25:19.467622+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:43:20.174095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341556+0000","last_change":"2023-03-03T06:25:13.837256+0000","last_active":"2023-03-03T06:25:18.341556+0000","last_peered":"2023-03-03T06:25:18.341556+0000","last_clean":"2023-03-03T06:25:18.341556+0000","last_became_active":"2023-03-03T06:25:13.836942+0000","last_became_peered":"2023-03-03T06:25:13.836942+0000","last_unstale":"2023-03-03T06:25:18.341556+0000","last_undegraded":"2023-03-03T06:25:18.341556+0000","last_fullsized":"2023-03-03T06:25:18.341556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:05:53.754239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357705+0000","last_change":"2023-03-03T06:25:07.204903+0000","last_active":"2023-03-03T06:25:18.357705+0000","last_peered":"2023-03-03T06:25:18.357705+0000","last_clean":"2023-03-03T06:25:18.357705+0000","last_became_active":"2023-03-03T06:25:07.204601+0000","last_became_peered":"2023-03-03T06:25:07.204601+0000","last_unstale":"2023-03-03T06:25:18.357705+0000","last_undegraded":"2023-03-03T06:25:18.357705+0000","last_fullsized":"2023-03-03T06:25:18.357705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:18:36.947912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343364+0000","last_change":"2023-03-03T06:25:13.304543+0000","last_active":"2023-03-03T06:25:18.343364+0000","last_peered":"2023-03-03T06:25:18.343364+0000","last_clean":"2023-03-03T06:25:18.343364+0000","last_became_active":"2023-03-03T06:25:13.304142+0000","last_became_peered":"2023-03-03T06:25:13.304142+0000","last_unstale":"2023-03-03T06:25:18.343364+0000","last_undegraded":"2023-03-03T06:25:18.343364+0000","last_fullsized":"2023-03-03T06:25:18.343364+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.876556+0000","last_change":"2023-03-03T06:25:18.354139+0000","last_active":"2023-03-03T06:25:18.876556+0000","last_peered":"2023-03-03T06:25:18.876556+0000","last_clean":"2023-03-03T06:25:18.876556+0000","last_became_active":"2023-03-03T06:25:18.353543+0000","last_became_peered":"2023-03-03T06:25:18.353543+0000","last_unstale":"2023-03-03T06:25:18.876556+0000","last_undegraded":"2023-03-03T06:25:18.876556+0000","last_fullsized":"2023-03-03T06:25:18.876556+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:11:23.940316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343906+0000","last_change":"2023-03-03T06:25:13.838819+0000","last_active":"2023-03-03T06:25:18.343906+0000","last_peered":"2023-03-03T06:25:18.343906+0000","last_clean":"2023-03-03T06:25:18.343906+0000","last_became_active":"2023-03-03T06:25:13.838558+0000","last_became_peered":"2023-03-03T06:25:13.838558+0000","last_unstale":"2023-03-03T06:25:18.343906+0000","last_undegraded":"2023-03-03T06:25:18.343906+0000","last_fullsized":"2023-03-03T06:25:18.343906+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350807+0000","last_change":"2023-03-03T06:25:07.208473+0000","last_active":"2023-03-03T06:25:18.350807+0000","last_peered":"2023-03-03T06:25:18.350807+0000","last_clean":"2023-03-03T06:25:18.350807+0000","last_became_active":"2023-03-03T06:25:07.208041+0000","last_became_peered":"2023-03-03T06:25:07.208041+0000","last_unstale":"2023-03-03T06:25:18.350807+0000","last_undegraded":"2023-03-03T06:25:18.350807+0000","last_fullsized":"2023-03-03T06:25:18.350807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:22:19.852493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357626+0000","last_change":"2023-03-03T06:25:13.308259+0000","last_active":"2023-03-03T06:25:18.357626+0000","last_peered":"2023-03-03T06:25:18.357626+0000","last_clean":"2023-03-03T06:25:18.357626+0000","last_became_active":"2023-03-03T06:25:13.307902+0000","last_became_peered":"2023-03-03T06:25:13.307902+0000","last_unstale":"2023-03-03T06:25:18.357626+0000","last_undegraded":"2023-03-03T06:25:18.357626+0000","last_fullsized":"2023-03-03T06:25:18.357626+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.459074+0000","last_change":"2023-03-03T06:25:18.355878+0000","last_active":"2023-03-03T06:25:19.459074+0000","last_peered":"2023-03-03T06:25:19.459074+0000","last_clean":"2023-03-03T06:25:19.459074+0000","last_became_active":"2023-03-03T06:25:18.355460+0000","last_became_peered":"2023-03-03T06:25:18.355460+0000","last_unstale":"2023-03-03T06:25:19.459074+0000","last_undegraded":"2023-03-03T06:25:19.459074+0000","last_fullsized":"2023-03-03T06:25:19.459074+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:38:51.150150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358771+0000","last_change":"2023-03-03T06:25:13.839947+0000","last_active":"2023-03-03T06:25:18.358771+0000","last_peered":"2023-03-03T06:25:18.358771+0000","last_clean":"2023-03-03T06:25:18.358771+0000","last_became_active":"2023-03-03T06:25:13.839632+0000","last_became_peered":"2023-03-03T06:25:13.839632+0000","last_unstale":"2023-03-03T06:25:18.358771+0000","last_undegraded":"2023-03-03T06:25:18.358771+0000","last_fullsized":"2023-03-03T06:25:18.358771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357780+0000","last_change":"2023-03-03T06:25:07.201865+0000","last_active":"2023-03-03T06:25:18.357780+0000","last_peered":"2023-03-03T06:25:18.357780+0000","last_clean":"2023-03-03T06:25:18.357780+0000","last_became_active":"2023-03-03T06:25:07.201411+0000","last_became_peered":"2023-03-03T06:25:07.201411+0000","last_unstale":"2023-03-03T06:25:18.357780+0000","last_undegraded":"2023-03-03T06:25:18.357780+0000","last_fullsized":"2023-03-03T06:25:18.357780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:17:53.442570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352970+0000","last_change":"2023-03-03T06:25:13.312426+0000","last_active":"2023-03-03T06:25:18.352970+0000","last_peered":"2023-03-03T06:25:18.352970+0000","last_clean":"2023-03-03T06:25:18.352970+0000","last_became_active":"2023-03-03T06:25:13.312129+0000","last_became_peered":"2023-03-03T06:25:13.312129+0000","last_unstale":"2023-03-03T06:25:18.352970+0000","last_undegraded":"2023-03-03T06:25:18.352970+0000","last_fullsized":"2023-03-03T06:25:18.352970+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.875532+0000","last_change":"2023-03-03T06:25:18.365085+0000","last_active":"2023-03-03T06:25:18.875532+0000","last_peered":"2023-03-03T06:25:18.875532+0000","last_clean":"2023-03-03T06:25:18.875532+0000","last_became_active":"2023-03-03T06:25:18.364690+0000","last_became_peered":"2023-03-03T06:25:18.364690+0000","last_unstale":"2023-03-03T06:25:18.875532+0000","last_undegraded":"2023-03-03T06:25:18.875532+0000","last_fullsized":"2023-03-03T06:25:18.875532+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:30.336080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357548+0000","last_change":"2023-03-03T06:25:13.828855+0000","last_active":"2023-03-03T06:25:18.357548+0000","last_peered":"2023-03-03T06:25:18.357548+0000","last_clean":"2023-03-03T06:25:18.357548+0000","last_became_active":"2023-03-03T06:25:13.828546+0000","last_became_peered":"2023-03-03T06:25:13.828546+0000","last_unstale":"2023-03-03T06:25:18.357548+0000","last_undegraded":"2023-03-03T06:25:18.357548+0000","last_fullsized":"2023-03-03T06:25:18.357548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334824+0000","last_change":"2023-03-03T06:25:13.298859+0000","last_active":"2023-03-03T06:25:18.334824+0000","last_peered":"2023-03-03T06:25:18.334824+0000","last_clean":"2023-03-03T06:25:18.334824+0000","last_became_active":"2023-03-03T06:25:13.298586+0000","last_became_peered":"2023-03-03T06:25:13.298586+0000","last_unstale":"2023-03-03T06:25:18.334824+0000","last_undegraded":"2023-03-03T06:25:18.334824+0000","last_fullsized":"2023-03-03T06:25:18.334824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:11:45.788884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353089+0000","last_change":"2023-03-03T06:25:07.201941+0000","last_active":"2023-03-03T06:25:18.353089+0000","last_peered":"2023-03-03T06:25:18.353089+0000","last_clean":"2023-03-03T06:25:18.353089+0000","last_became_active":"2023-03-03T06:25:07.201437+0000","last_became_peered":"2023-03-03T06:25:07.201437+0000","last_unstale":"2023-03-03T06:25:18.353089+0000","last_undegraded":"2023-03-03T06:25:18.353089+0000","last_fullsized":"2023-03-03T06:25:18.353089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:48:52.145810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.875719+0000","last_change":"2023-03-03T06:25:18.374636+0000","last_active":"2023-03-03T06:25:18.875719+0000","last_peered":"2023-03-03T06:25:18.875719+0000","last_clean":"2023-03-03T06:25:18.875719+0000","last_became_active":"2023-03-03T06:25:18.373925+0000","last_became_peered":"2023-03-03T06:25:18.373925+0000","last_unstale":"2023-03-03T06:25:18.875719+0000","last_undegraded":"2023-03-03T06:25:18.875719+0000","last_fullsized":"2023-03-03T06:25:18.875719+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:07:38.561763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.359409+0000","last_change":"2023-03-03T06:25:13.833104+0000","last_active":"2023-03-03T06:25:18.359409+0000","last_peered":"2023-03-03T06:25:18.359409+0000","last_clean":"2023-03-03T06:25:18.359409+0000","last_became_active":"2023-03-03T06:25:13.832815+0000","last_became_peered":"2023-03-03T06:25:13.832815+0000","last_unstale":"2023-03-03T06:25:18.359409+0000","last_undegraded":"2023-03-03T06:25:18.359409+0000","last_fullsized":"2023-03-03T06:25:18.359409+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.356972+0000","last_change":"2023-03-03T06:25:13.298339+0000","last_active":"2023-03-03T06:25:18.356972+0000","last_peered":"2023-03-03T06:25:18.356972+0000","last_clean":"2023-03-03T06:25:18.356972+0000","last_became_active":"2023-03-03T06:25:13.297116+0000","last_became_peered":"2023-03-03T06:25:13.297116+0000","last_unstale":"2023-03-03T06:25:18.356972+0000","last_undegraded":"2023-03-03T06:25:18.356972+0000","last_fullsized":"2023-03-03T06:25:18.356972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335947+0000","last_change":"2023-03-03T06:25:07.437275+0000","last_active":"2023-03-03T06:25:18.335947+0000","last_peered":"2023-03-03T06:25:18.335947+0000","last_clean":"2023-03-03T06:25:18.335947+0000","last_became_active":"2023-03-03T06:25:07.436823+0000","last_became_peered":"2023-03-03T06:25:07.436823+0000","last_unstale":"2023-03-03T06:25:18.335947+0000","last_undegraded":"2023-03-03T06:25:18.335947+0000","last_fullsized":"2023-03-03T06:25:18.335947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:24:15.529041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217928+0000","last_change":"2023-03-03T06:25:18.361949+0000","last_active":"2023-03-03T06:25:19.217928+0000","last_peered":"2023-03-03T06:25:19.217928+0000","last_clean":"2023-03-03T06:25:19.217928+0000","last_became_active":"2023-03-03T06:25:18.361407+0000","last_became_peered":"2023-03-03T06:25:18.361407+0000","last_unstale":"2023-03-03T06:25:19.217928+0000","last_undegraded":"2023-03-03T06:25:19.217928+0000","last_fullsized":"2023-03-03T06:25:19.217928+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:59.071940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344511+0000","last_change":"2023-03-03T06:25:13.833788+0000","last_active":"2023-03-03T06:25:18.344511+0000","last_peered":"2023-03-03T06:25:18.344511+0000","last_clean":"2023-03-03T06:25:18.344511+0000","last_became_active":"2023-03-03T06:25:13.833532+0000","last_became_peered":"2023-03-03T06:25:13.833532+0000","last_unstale":"2023-03-03T06:25:18.344511+0000","last_undegraded":"2023-03-03T06:25:18.344511+0000","last_fullsized":"2023-03-03T06:25:18.344511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:59:50.232210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342831+0000","last_change":"2023-03-03T06:25:13.308357+0000","last_active":"2023-03-03T06:25:18.342831+0000","last_peered":"2023-03-03T06:25:18.342831+0000","last_clean":"2023-03-03T06:25:18.342831+0000","last_became_active":"2023-03-03T06:25:13.307981+0000","last_became_peered":"2023-03-03T06:25:13.307981+0000","last_unstale":"2023-03-03T06:25:18.342831+0000","last_undegraded":"2023-03-03T06:25:18.342831+0000","last_fullsized":"2023-03-03T06:25:18.342831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346873+0000","last_change":"2023-03-03T06:25:07.208585+0000","last_active":"2023-03-03T06:25:18.346873+0000","last_peered":"2023-03-03T06:25:18.346873+0000","last_clean":"2023-03-03T06:25:18.346873+0000","last_became_active":"2023-03-03T06:25:07.208257+0000","last_became_peered":"2023-03-03T06:25:07.208257+0000","last_unstale":"2023-03-03T06:25:18.346873+0000","last_undegraded":"2023-03-03T06:25:18.346873+0000","last_fullsized":"2023-03-03T06:25:18.346873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:11:52.893733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879358+0000","last_change":"2023-03-03T06:25:18.369960+0000","last_active":"2023-03-03T06:25:18.879358+0000","last_peered":"2023-03-03T06:25:18.879358+0000","last_clean":"2023-03-03T06:25:18.879358+0000","last_became_active":"2023-03-03T06:25:18.367846+0000","last_became_peered":"2023-03-03T06:25:18.367846+0000","last_unstale":"2023-03-03T06:25:18.879358+0000","last_undegraded":"2023-03-03T06:25:18.879358+0000","last_fullsized":"2023-03-03T06:25:18.879358+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:12:09.571057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342233+0000","last_change":"2023-03-03T06:25:13.857729+0000","last_active":"2023-03-03T06:25:18.342233+0000","last_peered":"2023-03-03T06:25:18.342233+0000","last_clean":"2023-03-03T06:25:18.342233+0000","last_became_active":"2023-03-03T06:25:13.857453+0000","last_became_peered":"2023-03-03T06:25:13.857453+0000","last_unstale":"2023-03-03T06:25:18.342233+0000","last_undegraded":"2023-03-03T06:25:18.342233+0000","last_fullsized":"2023-03-03T06:25:18.342233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:46:08.313482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.339103+0000","last_change":"2023-03-03T06:25:13.303419+0000","last_active":"2023-03-03T06:25:18.339103+0000","last_peered":"2023-03-03T06:25:18.339103+0000","last_clean":"2023-03-03T06:25:18.339103+0000","last_became_active":"2023-03-03T06:25:13.303084+0000","last_became_peered":"2023-03-03T06:25:13.303084+0000","last_unstale":"2023-03-03T06:25:18.339103+0000","last_undegraded":"2023-03-03T06:25:18.339103+0000","last_fullsized":"2023-03-03T06:25:18.339103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352430+0000","last_change":"2023-03-03T06:25:07.203115+0000","last_active":"2023-03-03T06:25:18.352430+0000","last_peered":"2023-03-03T06:25:18.352430+0000","last_clean":"2023-03-03T06:25:18.352430+0000","last_became_active":"2023-03-03T06:25:07.202705+0000","last_became_peered":"2023-03-03T06:25:07.202705+0000","last_unstale":"2023-03-03T06:25:18.352430+0000","last_undegraded":"2023-03-03T06:25:18.352430+0000","last_fullsized":"2023-03-03T06:25:18.352430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:11:32.938698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346951+0000","last_change":"2023-03-03T06:25:02.138688+0000","last_active":"2023-03-03T06:25:18.346951+0000","last_peered":"2023-03-03T06:25:18.346951+0000","last_clean":"2023-03-03T06:25:18.346951+0000","last_became_active":"2023-03-03T06:25:02.137945+0000","last_became_peered":"2023-03-03T06:25:02.137945+0000","last_unstale":"2023-03-03T06:25:18.346951+0000","last_undegraded":"2023-03-03T06:25:18.346951+0000","last_fullsized":"2023-03-03T06:25:18.346951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:00.679233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:00.679233+0000","last_clean_scrub_stamp":"2023-03-03T06:25:00.679233+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:24:46.304379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.463093+0000","last_change":"2023-03-03T06:25:18.368371+0000","last_active":"2023-03-03T06:25:19.463093+0000","last_peered":"2023-03-03T06:25:19.463093+0000","last_clean":"2023-03-03T06:25:19.463093+0000","last_became_active":"2023-03-03T06:25:18.367896+0000","last_became_peered":"2023-03-03T06:25:18.367896+0000","last_unstale":"2023-03-03T06:25:19.463093+0000","last_undegraded":"2023-03-03T06:25:19.463093+0000","last_fullsized":"2023-03-03T06:25:19.463093+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:43:20.174095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358378+0000","last_change":"2023-03-03T06:25:13.842311+0000","last_active":"2023-03-03T06:25:18.358378+0000","last_peered":"2023-03-03T06:25:18.358378+0000","last_clean":"2023-03-03T06:25:18.358378+0000","last_became_active":"2023-03-03T06:25:13.841952+0000","last_became_peered":"2023-03-03T06:25:13.841952+0000","last_unstale":"2023-03-03T06:25:18.358378+0000","last_undegraded":"2023-03-03T06:25:18.358378+0000","last_fullsized":"2023-03-03T06:25:18.358378+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:46:08.313482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341679+0000","last_change":"2023-03-03T06:25:13.306584+0000","last_active":"2023-03-03T06:25:18.341679+0000","last_peered":"2023-03-03T06:25:18.341679+0000","last_clean":"2023-03-03T06:25:18.341679+0000","last_became_active":"2023-03-03T06:25:13.306295+0000","last_became_peered":"2023-03-03T06:25:13.306295+0000","last_unstale":"2023-03-03T06:25:18.341679+0000","last_undegraded":"2023-03-03T06:25:18.341679+0000","last_fullsized":"2023-03-03T06:25:18.341679+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.878155+0000","last_change":"2023-03-03T06:25:18.361381+0000","last_active":"2023-03-03T06:25:18.878155+0000","last_peered":"2023-03-03T06:25:18.878155+0000","last_clean":"2023-03-03T06:25:18.878155+0000","last_became_active":"2023-03-03T06:25:18.360819+0000","last_became_peered":"2023-03-03T06:25:18.360819+0000","last_unstale":"2023-03-03T06:25:18.878155+0000","last_undegraded":"2023-03-03T06:25:18.878155+0000","last_fullsized":"2023-03-03T06:25:18.878155+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:39:21.927192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335650+0000","last_change":"2023-03-03T06:25:13.848629+0000","last_active":"2023-03-03T06:25:18.335650+0000","last_peered":"2023-03-03T06:25:18.335650+0000","last_clean":"2023-03-03T06:25:18.335650+0000","last_became_active":"2023-03-03T06:25:13.848353+0000","last_became_peered":"2023-03-03T06:25:13.848353+0000","last_unstale":"2023-03-03T06:25:18.335650+0000","last_undegraded":"2023-03-03T06:25:18.335650+0000","last_fullsized":"2023-03-03T06:25:18.335650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:59:50.232210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335310+0000","last_change":"2023-03-03T06:25:13.297292+0000","last_active":"2023-03-03T06:25:18.335310+0000","last_peered":"2023-03-03T06:25:18.335310+0000","last_clean":"2023-03-03T06:25:18.335310+0000","last_became_active":"2023-03-03T06:25:13.296692+0000","last_became_peered":"2023-03-03T06:25:13.296692+0000","last_unstale":"2023-03-03T06:25:18.335310+0000","last_undegraded":"2023-03-03T06:25:18.335310+0000","last_fullsized":"2023-03-03T06:25:18.335310+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:09:43.297428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879361+0000","last_change":"2023-03-03T06:25:18.369017+0000","last_active":"2023-03-03T06:25:18.879361+0000","last_peered":"2023-03-03T06:25:18.879361+0000","last_clean":"2023-03-03T06:25:18.879361+0000","last_became_active":"2023-03-03T06:25:18.366873+0000","last_became_peered":"2023-03-03T06:25:18.366873+0000","last_unstale":"2023-03-03T06:25:18.879361+0000","last_undegraded":"2023-03-03T06:25:18.879361+0000","last_fullsized":"2023-03-03T06:25:18.879361+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:17:02.442524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334965+0000","last_change":"2023-03-03T06:25:13.855863+0000","last_active":"2023-03-03T06:25:18.334965+0000","last_peered":"2023-03-03T06:25:18.334965+0000","last_clean":"2023-03-03T06:25:18.334965+0000","last_became_active":"2023-03-03T06:25:13.855565+0000","last_became_peered":"2023-03-03T06:25:13.855565+0000","last_unstale":"2023-03-03T06:25:18.334965+0000","last_undegraded":"2023-03-03T06:25:18.334965+0000","last_fullsized":"2023-03-03T06:25:18.334965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:45:26.049658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341408+0000","last_change":"2023-03-03T06:25:13.306690+0000","last_active":"2023-03-03T06:25:18.341408+0000","last_peered":"2023-03-03T06:25:18.341408+0000","last_clean":"2023-03-03T06:25:18.341408+0000","last_became_active":"2023-03-03T06:25:13.306348+0000","last_became_peered":"2023-03-03T06:25:13.306348+0000","last_unstale":"2023-03-03T06:25:18.341408+0000","last_undegraded":"2023-03-03T06:25:18.341408+0000","last_fullsized":"2023-03-03T06:25:18.341408+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461865+0000","last_change":"2023-03-03T06:25:18.373926+0000","last_active":"2023-03-03T06:25:19.461865+0000","last_peered":"2023-03-03T06:25:19.461865+0000","last_clean":"2023-03-03T06:25:19.461865+0000","last_became_active":"2023-03-03T06:25:18.373371+0000","last_became_peered":"2023-03-03T06:25:18.373371+0000","last_unstale":"2023-03-03T06:25:19.461865+0000","last_undegraded":"2023-03-03T06:25:19.461865+0000","last_fullsized":"2023-03-03T06:25:19.461865+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:17:02.442524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335386+0000","last_change":"2023-03-03T06:25:13.846359+0000","last_active":"2023-03-03T06:25:18.335386+0000","last_peered":"2023-03-03T06:25:18.335386+0000","last_clean":"2023-03-03T06:25:18.335386+0000","last_became_active":"2023-03-03T06:25:13.845966+0000","last_became_peered":"2023-03-03T06:25:13.845966+0000","last_unstale":"2023-03-03T06:25:18.335386+0000","last_undegraded":"2023-03-03T06:25:18.335386+0000","last_fullsized":"2023-03-03T06:25:18.335386+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:38:43.138896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341996+0000","last_change":"2023-03-03T06:25:13.310274+0000","last_active":"2023-03-03T06:25:18.341996+0000","last_peered":"2023-03-03T06:25:18.341996+0000","last_clean":"2023-03-03T06:25:18.341996+0000","last_became_active":"2023-03-03T06:25:13.310005+0000","last_became_peered":"2023-03-03T06:25:13.310005+0000","last_unstale":"2023-03-03T06:25:18.341996+0000","last_undegraded":"2023-03-03T06:25:18.341996+0000","last_fullsized":"2023-03-03T06:25:18.341996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217857+0000","last_change":"2023-03-03T06:25:18.362444+0000","last_active":"2023-03-03T06:25:19.217857+0000","last_peered":"2023-03-03T06:25:19.217857+0000","last_clean":"2023-03-03T06:25:19.217857+0000","last_became_active":"2023-03-03T06:25:18.362087+0000","last_became_peered":"2023-03-03T06:25:18.362087+0000","last_unstale":"2023-03-03T06:25:19.217857+0000","last_undegraded":"2023-03-03T06:25:19.217857+0000","last_fullsized":"2023-03-03T06:25:19.217857+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:52:17.597685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343862+0000","last_change":"2023-03-03T06:25:13.834113+0000","last_active":"2023-03-03T06:25:18.343862+0000","last_peered":"2023-03-03T06:25:18.343862+0000","last_clean":"2023-03-03T06:25:18.343862+0000","last_became_active":"2023-03-03T06:25:13.833874+0000","last_became_peered":"2023-03-03T06:25:13.833874+0000","last_unstale":"2023-03-03T06:25:18.343862+0000","last_undegraded":"2023-03-03T06:25:18.343862+0000","last_fullsized":"2023-03-03T06:25:18.343862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352647+0000","last_change":"2023-03-03T06:25:13.310942+0000","last_active":"2023-03-03T06:25:18.352647+0000","last_peered":"2023-03-03T06:25:18.352647+0000","last_clean":"2023-03-03T06:25:18.352647+0000","last_became_active":"2023-03-03T06:25:13.310632+0000","last_became_peered":"2023-03-03T06:25:13.310632+0000","last_unstale":"2023-03-03T06:25:18.352647+0000","last_undegraded":"2023-03-03T06:25:18.352647+0000","last_fullsized":"2023-03-03T06:25:18.352647+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:16:16.586666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217181+0000","last_change":"2023-03-03T06:25:18.360063+0000","last_active":"2023-03-03T06:25:19.217181+0000","last_peered":"2023-03-03T06:25:19.217181+0000","last_clean":"2023-03-03T06:25:19.217181+0000","last_became_active":"2023-03-03T06:25:18.359549+0000","last_became_peered":"2023-03-03T06:25:18.359549+0000","last_unstale":"2023-03-03T06:25:19.217181+0000","last_undegraded":"2023-03-03T06:25:19.217181+0000","last_fullsized":"2023-03-03T06:25:19.217181+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:30:28.010892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341203+0000","last_change":"2023-03-03T06:25:13.844840+0000","last_active":"2023-03-03T06:25:18.341203+0000","last_peered":"2023-03-03T06:25:18.341203+0000","last_clean":"2023-03-03T06:25:18.341203+0000","last_became_active":"2023-03-03T06:25:13.844556+0000","last_became_peered":"2023-03-03T06:25:13.844556+0000","last_unstale":"2023-03-03T06:25:18.341203+0000","last_undegraded":"2023-03-03T06:25:18.341203+0000","last_fullsized":"2023-03-03T06:25:18.341203+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:19:49.547542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350702+0000","last_change":"2023-03-03T06:25:13.303131+0000","last_active":"2023-03-03T06:25:18.350702+0000","last_peered":"2023-03-03T06:25:18.350702+0000","last_clean":"2023-03-03T06:25:18.350702+0000","last_became_active":"2023-03-03T06:25:13.302861+0000","last_became_peered":"2023-03-03T06:25:13.302861+0000","last_unstale":"2023-03-03T06:25:18.350702+0000","last_undegraded":"2023-03-03T06:25:18.350702+0000","last_fullsized":"2023-03-03T06:25:18.350702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:46:07.297374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217444+0000","last_change":"2023-03-03T06:25:18.363570+0000","last_active":"2023-03-03T06:25:19.217444+0000","last_peered":"2023-03-03T06:25:19.217444+0000","last_clean":"2023-03-03T06:25:19.217444+0000","last_became_active":"2023-03-03T06:25:18.363224+0000","last_became_peered":"2023-03-03T06:25:18.363224+0000","last_unstale":"2023-03-03T06:25:19.217444+0000","last_undegraded":"2023-03-03T06:25:19.217444+0000","last_fullsized":"2023-03-03T06:25:19.217444+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:38:51.150150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335118+0000","last_change":"2023-03-03T06:25:13.837508+0000","last_active":"2023-03-03T06:25:18.335118+0000","last_peered":"2023-03-03T06:25:18.335118+0000","last_clean":"2023-03-03T06:25:18.335118+0000","last_became_active":"2023-03-03T06:25:13.836864+0000","last_became_peered":"2023-03-03T06:25:13.836864+0000","last_unstale":"2023-03-03T06:25:18.335118+0000","last_undegraded":"2023-03-03T06:25:18.335118+0000","last_fullsized":"2023-03-03T06:25:18.335118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:33:19.697671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350575+0000","last_change":"2023-03-03T06:25:13.308653+0000","last_active":"2023-03-03T06:25:18.350575+0000","last_peered":"2023-03-03T06:25:18.350575+0000","last_clean":"2023-03-03T06:25:18.350575+0000","last_became_active":"2023-03-03T06:25:13.308404+0000","last_became_peered":"2023-03-03T06:25:13.308404+0000","last_unstale":"2023-03-03T06:25:18.350575+0000","last_undegraded":"2023-03-03T06:25:18.350575+0000","last_fullsized":"2023-03-03T06:25:18.350575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:42:29.976451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217056+0000","last_change":"2023-03-03T06:25:18.369545+0000","last_active":"2023-03-03T06:25:19.217056+0000","last_peered":"2023-03-03T06:25:19.217056+0000","last_clean":"2023-03-03T06:25:19.217056+0000","last_became_active":"2023-03-03T06:25:18.369075+0000","last_became_peered":"2023-03-03T06:25:18.369075+0000","last_unstale":"2023-03-03T06:25:19.217056+0000","last_undegraded":"2023-03-03T06:25:19.217056+0000","last_fullsized":"2023-03-03T06:25:19.217056+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:45:30.096214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352873+0000","last_change":"2023-03-03T06:25:13.836258+0000","last_active":"2023-03-03T06:25:18.352873+0000","last_peered":"2023-03-03T06:25:18.352873+0000","last_clean":"2023-03-03T06:25:18.352873+0000","last_became_active":"2023-03-03T06:25:13.835993+0000","last_became_peered":"2023-03-03T06:25:13.835993+0000","last_unstale":"2023-03-03T06:25:18.352873+0000","last_undegraded":"2023-03-03T06:25:18.352873+0000","last_fullsized":"2023-03-03T06:25:18.352873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350886+0000","last_change":"2023-03-03T06:25:13.307887+0000","last_active":"2023-03-03T06:25:18.350886+0000","last_peered":"2023-03-03T06:25:18.350886+0000","last_clean":"2023-03-03T06:25:18.350886+0000","last_became_active":"2023-03-03T06:25:13.307622+0000","last_became_peered":"2023-03-03T06:25:13.307622+0000","last_unstale":"2023-03-03T06:25:18.350886+0000","last_undegraded":"2023-03-03T06:25:18.350886+0000","last_fullsized":"2023-03-03T06:25:18.350886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:51:44.342844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.462959+0000","last_change":"2023-03-03T06:25:18.366887+0000","last_active":"2023-03-03T06:25:19.462959+0000","last_peered":"2023-03-03T06:25:19.462959+0000","last_clean":"2023-03-03T06:25:19.462959+0000","last_became_active":"2023-03-03T06:25:18.366248+0000","last_became_peered":"2023-03-03T06:25:18.366248+0000","last_unstale":"2023-03-03T06:25:19.462959+0000","last_undegraded":"2023-03-03T06:25:19.462959+0000","last_fullsized":"2023-03-03T06:25:19.462959+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:06:08.404601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879274+0000","last_change":"2023-03-03T06:25:18.374976+0000","last_active":"2023-03-03T06:25:18.879274+0000","last_peered":"2023-03-03T06:25:18.879274+0000","last_clean":"2023-03-03T06:25:18.879274+0000","last_became_active":"2023-03-03T06:25:18.374414+0000","last_became_peered":"2023-03-03T06:25:18.374414+0000","last_unstale":"2023-03-03T06:25:18.879274+0000","last_undegraded":"2023-03-03T06:25:18.879274+0000","last_fullsized":"2023-03-03T06:25:18.879274+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:52:08.870718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.340487+0000","last_change":"2023-03-03T06:25:13.297905+0000","last_active":"2023-03-03T06:25:18.340487+0000","last_peered":"2023-03-03T06:25:18.340487+0000","last_clean":"2023-03-03T06:25:18.340487+0000","last_became_active":"2023-03-03T06:25:13.297568+0000","last_became_peered":"2023-03-03T06:25:13.297568+0000","last_unstale":"2023-03-03T06:25:18.340487+0000","last_undegraded":"2023-03-03T06:25:18.340487+0000","last_fullsized":"2023-03-03T06:25:18.340487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:17:34.388274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343217+0000","last_change":"2023-03-03T06:25:13.851715+0000","last_active":"2023-03-03T06:25:18.343217+0000","last_peered":"2023-03-03T06:25:18.343217+0000","last_clean":"2023-03-03T06:25:18.343217+0000","last_became_active":"2023-03-03T06:25:13.851377+0000","last_became_peered":"2023-03-03T06:25:13.851377+0000","last_unstale":"2023-03-03T06:25:18.343217+0000","last_undegraded":"2023-03-03T06:25:18.343217+0000","last_fullsized":"2023-03-03T06:25:18.343217+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467810+0000","last_change":"2023-03-03T06:25:18.356507+0000","last_active":"2023-03-03T06:25:19.467810+0000","last_peered":"2023-03-03T06:25:19.467810+0000","last_clean":"2023-03-03T06:25:19.467810+0000","last_became_active":"2023-03-03T06:25:18.356097+0000","last_became_peered":"2023-03-03T06:25:18.356097+0000","last_unstale":"2023-03-03T06:25:19.467810+0000","last_undegraded":"2023-03-03T06:25:19.467810+0000","last_fullsized":"2023-03-03T06:25:19.467810+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:59:33.700378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.351468+0000","last_change":"2023-03-03T06:25:13.306049+0000","last_active":"2023-03-03T06:25:18.351468+0000","last_peered":"2023-03-03T06:25:18.351468+0000","last_clean":"2023-03-03T06:25:18.351468+0000","last_became_active":"2023-03-03T06:25:13.305742+0000","last_became_peered":"2023-03-03T06:25:13.305742+0000","last_unstale":"2023-03-03T06:25:18.351468+0000","last_undegraded":"2023-03-03T06:25:18.351468+0000","last_fullsized":"2023-03-03T06:25:18.351468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:29:05.784203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344608+0000","last_change":"2023-03-03T06:25:13.841823+0000","last_active":"2023-03-03T06:25:18.344608+0000","last_peered":"2023-03-03T06:25:18.344608+0000","last_clean":"2023-03-03T06:25:18.344608+0000","last_became_active":"2023-03-03T06:25:13.841581+0000","last_became_peered":"2023-03-03T06:25:13.841581+0000","last_unstale":"2023-03-03T06:25:18.344608+0000","last_undegraded":"2023-03-03T06:25:18.344608+0000","last_fullsized":"2023-03-03T06:25:18.344608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342620+0000","last_change":"2023-03-03T06:25:13.307869+0000","last_active":"2023-03-03T06:25:18.342620+0000","last_peered":"2023-03-03T06:25:18.342620+0000","last_clean":"2023-03-03T06:25:18.342620+0000","last_became_active":"2023-03-03T06:25:13.303371+0000","last_became_peered":"2023-03-03T06:25:13.303371+0000","last_unstale":"2023-03-03T06:25:18.342620+0000","last_undegraded":"2023-03-03T06:25:18.342620+0000","last_fullsized":"2023-03-03T06:25:18.342620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352774+0000","last_change":"2023-03-03T06:25:13.853988+0000","last_active":"2023-03-03T06:25:18.352774+0000","last_peered":"2023-03-03T06:25:18.352774+0000","last_clean":"2023-03-03T06:25:18.352774+0000","last_became_active":"2023-03-03T06:25:13.853714+0000","last_became_peered":"2023-03-03T06:25:13.853714+0000","last_unstale":"2023-03-03T06:25:18.352774+0000","last_undegraded":"2023-03-03T06:25:18.352774+0000","last_fullsized":"2023-03-03T06:25:18.352774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:51:30.984078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467763+0000","last_change":"2023-03-03T06:25:18.372232+0000","last_active":"2023-03-03T06:25:19.467763+0000","last_peered":"2023-03-03T06:25:19.467763+0000","last_clean":"2023-03-03T06:25:19.467763+0000","last_became_active":"2023-03-03T06:25:18.371809+0000","last_became_peered":"2023-03-03T06:25:18.371809+0000","last_unstale":"2023-03-03T06:25:19.467763+0000","last_undegraded":"2023-03-03T06:25:19.467763+0000","last_fullsized":"2023-03-03T06:25:19.467763+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:11:23.940316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346535+0000","last_change":"2023-03-03T06:25:13.307069+0000","last_active":"2023-03-03T06:25:18.346535+0000","last_peered":"2023-03-03T06:25:18.346535+0000","last_clean":"2023-03-03T06:25:18.346535+0000","last_became_active":"2023-03-03T06:25:13.306791+0000","last_became_peered":"2023-03-03T06:25:13.306791+0000","last_unstale":"2023-03-03T06:25:18.346535+0000","last_undegraded":"2023-03-03T06:25:18.346535+0000","last_fullsized":"2023-03-03T06:25:18.346535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:19:48.531435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343519+0000","last_change":"2023-03-03T06:25:13.830026+0000","last_active":"2023-03-03T06:25:18.343519+0000","last_peered":"2023-03-03T06:25:18.343519+0000","last_clean":"2023-03-03T06:25:18.343519+0000","last_became_active":"2023-03-03T06:25:13.829773+0000","last_became_peered":"2023-03-03T06:25:13.829773+0000","last_unstale":"2023-03-03T06:25:18.343519+0000","last_undegraded":"2023-03-03T06:25:18.343519+0000","last_fullsized":"2023-03-03T06:25:18.343519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:35:57.127264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879167+0000","last_change":"2023-03-03T06:25:18.374930+0000","last_active":"2023-03-03T06:25:18.879167+0000","last_peered":"2023-03-03T06:25:18.879167+0000","last_clean":"2023-03-03T06:25:18.879167+0000","last_became_active":"2023-03-03T06:25:18.374374+0000","last_became_peered":"2023-03-03T06:25:18.374374+0000","last_unstale":"2023-03-03T06:25:18.879167+0000","last_undegraded":"2023-03-03T06:25:18.879167+0000","last_fullsized":"2023-03-03T06:25:18.879167+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:09:37.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.340359+0000","last_change":"2023-03-03T06:25:13.840311+0000","last_active":"2023-03-03T06:25:18.340359+0000","last_peered":"2023-03-03T06:25:18.340359+0000","last_clean":"2023-03-03T06:25:18.340359+0000","last_became_active":"2023-03-03T06:25:13.839648+0000","last_became_peered":"2023-03-03T06:25:13.839648+0000","last_unstale":"2023-03-03T06:25:18.340359+0000","last_undegraded":"2023-03-03T06:25:18.340359+0000","last_fullsized":"2023-03-03T06:25:18.340359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:29:06.800310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337721+0000","last_change":"2023-03-03T06:25:13.836213+0000","last_active":"2023-03-03T06:25:18.337721+0000","last_peered":"2023-03-03T06:25:18.337721+0000","last_clean":"2023-03-03T06:25:18.337721+0000","last_became_active":"2023-03-03T06:25:13.835890+0000","last_became_peered":"2023-03-03T06:25:13.835890+0000","last_unstale":"2023-03-03T06:25:18.337721+0000","last_undegraded":"2023-03-03T06:25:18.337721+0000","last_fullsized":"2023-03-03T06:25:18.337721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:55.025383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335018+0000","last_change":"2023-03-03T06:25:13.834786+0000","last_active":"2023-03-03T06:25:18.335018+0000","last_peered":"2023-03-03T06:25:18.335018+0000","last_clean":"2023-03-03T06:25:18.335018+0000","last_became_active":"2023-03-03T06:25:13.834531+0000","last_became_peered":"2023-03-03T06:25:13.834531+0000","last_unstale":"2023-03-03T06:25:18.335018+0000","last_undegraded":"2023-03-03T06:25:18.335018+0000","last_fullsized":"2023-03-03T06:25:18.335018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:07:17.177238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342130+0000","last_change":"2023-03-03T06:25:13.850334+0000","last_active":"2023-03-03T06:25:18.342130+0000","last_peered":"2023-03-03T06:25:18.342130+0000","last_clean":"2023-03-03T06:25:18.342130+0000","last_became_active":"2023-03-03T06:25:13.850082+0000","last_became_peered":"2023-03-03T06:25:13.850082+0000","last_unstale":"2023-03-03T06:25:18.342130+0000","last_undegraded":"2023-03-03T06:25:18.342130+0000","last_fullsized":"2023-03-03T06:25:18.342130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357706+0000","last_change":"2023-03-03T06:25:13.850690+0000","last_active":"2023-03-03T06:25:18.357706+0000","last_peered":"2023-03-03T06:25:18.357706+0000","last_clean":"2023-03-03T06:25:18.357706+0000","last_became_active":"2023-03-03T06:25:13.850392+0000","last_became_peered":"2023-03-03T06:25:13.850392+0000","last_unstale":"2023-03-03T06:25:18.357706+0000","last_undegraded":"2023-03-03T06:25:18.357706+0000","last_fullsized":"2023-03-03T06:25:18.357706+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352673+0000","last_change":"2023-03-03T06:25:13.853965+0000","last_active":"2023-03-03T06:25:18.352673+0000","last_peered":"2023-03-03T06:25:18.352673+0000","last_clean":"2023-03-03T06:25:18.352673+0000","last_became_active":"2023-03-03T06:25:13.853689+0000","last_became_peered":"2023-03-03T06:25:13.853689+0000","last_unstale":"2023-03-03T06:25:18.352673+0000","last_undegraded":"2023-03-03T06:25:18.352673+0000","last_fullsized":"2023-03-03T06:25:18.352673+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:09:17.786520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357579+0000","last_change":"2023-03-03T06:25:13.828945+0000","last_active":"2023-03-03T06:25:18.357579+0000","last_peered":"2023-03-03T06:25:18.357579+0000","last_clean":"2023-03-03T06:25:18.357579+0000","last_became_active":"2023-03-03T06:25:13.828655+0000","last_became_peered":"2023-03-03T06:25:13.828655+0000","last_unstale":"2023-03-03T06:25:18.357579+0000","last_undegraded":"2023-03-03T06:25:18.357579+0000","last_fullsized":"2023-03-03T06:25:18.357579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:51:30.984078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358304+0000","last_change":"2023-03-03T06:25:13.834307+0000","last_active":"2023-03-03T06:25:18.358304+0000","last_peered":"2023-03-03T06:25:18.358304+0000","last_clean":"2023-03-03T06:25:18.358304+0000","last_became_active":"2023-03-03T06:25:13.834054+0000","last_became_peered":"2023-03-03T06:25:13.834054+0000","last_unstale":"2023-03-03T06:25:18.358304+0000","last_undegraded":"2023-03-03T06:25:18.358304+0000","last_fullsized":"2023-03-03T06:25:18.358304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:54:02.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346796+0000","last_change":"2023-03-03T06:25:13.856830+0000","last_active":"2023-03-03T06:25:18.346796+0000","last_peered":"2023-03-03T06:25:18.346796+0000","last_clean":"2023-03-03T06:25:18.346796+0000","last_became_active":"2023-03-03T06:25:13.856494+0000","last_became_peered":"2023-03-03T06:25:13.856494+0000","last_unstale":"2023-03-03T06:25:18.346796+0000","last_undegraded":"2023-03-03T06:25:18.346796+0000","last_fullsized":"2023-03-03T06:25:18.346796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:59:50.232210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337609+0000","last_change":"2023-03-03T06:25:13.836186+0000","last_active":"2023-03-03T06:25:18.337609+0000","last_peered":"2023-03-03T06:25:18.337609+0000","last_clean":"2023-03-03T06:25:18.337609+0000","last_became_active":"2023-03-03T06:25:13.835481+0000","last_became_peered":"2023-03-03T06:25:13.835481+0000","last_unstale":"2023-03-03T06:25:18.337609+0000","last_undegraded":"2023-03-03T06:25:18.337609+0000","last_fullsized":"2023-03-03T06:25:18.337609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:30:23.964335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342530+0000","last_change":"2023-03-03T06:25:13.834687+0000","last_active":"2023-03-03T06:25:18.342530+0000","last_peered":"2023-03-03T06:25:18.342530+0000","last_clean":"2023-03-03T06:25:18.342530+0000","last_became_active":"2023-03-03T06:25:13.834424+0000","last_became_peered":"2023-03-03T06:25:13.834424+0000","last_unstale":"2023-03-03T06:25:18.342530+0000","last_undegraded":"2023-03-03T06:25:18.342530+0000","last_fullsized":"2023-03-03T06:25:18.342530+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346438+0000","last_change":"2023-03-03T06:25:13.855849+0000","last_active":"2023-03-03T06:25:18.346438+0000","last_peered":"2023-03-03T06:25:18.346438+0000","last_clean":"2023-03-03T06:25:18.346438+0000","last_became_active":"2023-03-03T06:25:13.855609+0000","last_became_peered":"2023-03-03T06:25:13.855609+0000","last_unstale":"2023-03-03T06:25:18.346438+0000","last_undegraded":"2023-03-03T06:25:18.346438+0000","last_fullsized":"2023-03-03T06:25:18.346438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335348+0000","last_change":"2023-03-03T06:25:13.846670+0000","last_active":"2023-03-03T06:25:18.335348+0000","last_peered":"2023-03-03T06:25:18.335348+0000","last_clean":"2023-03-03T06:25:18.335348+0000","last_became_active":"2023-03-03T06:25:13.846349+0000","last_became_peered":"2023-03-03T06:25:13.846349+0000","last_unstale":"2023-03-03T06:25:18.335348+0000","last_undegraded":"2023-03-03T06:25:18.335348+0000","last_fullsized":"2023-03-03T06:25:18.335348+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:30:23.964335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337310+0000","last_change":"2023-03-03T06:25:13.839483+0000","last_active":"2023-03-03T06:25:18.337310+0000","last_peered":"2023-03-03T06:25:18.337310+0000","last_clean":"2023-03-03T06:25:18.337310+0000","last_became_active":"2023-03-03T06:25:13.838497+0000","last_became_peered":"2023-03-03T06:25:13.838497+0000","last_unstale":"2023-03-03T06:25:18.337310+0000","last_undegraded":"2023-03-03T06:25:18.337310+0000","last_fullsized":"2023-03-03T06:25:18.337310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:08.122967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334979+0000","last_change":"2023-03-03T06:25:13.855262+0000","last_active":"2023-03-03T06:25:18.334979+0000","last_peered":"2023-03-03T06:25:18.334979+0000","last_clean":"2023-03-03T06:25:18.334979+0000","last_became_active":"2023-03-03T06:25:13.848660+0000","last_became_peered":"2023-03-03T06:25:13.848660+0000","last_unstale":"2023-03-03T06:25:18.334979+0000","last_undegraded":"2023-03-03T06:25:18.334979+0000","last_fullsized":"2023-03-03T06:25:18.334979+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:55.025383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344570+0000","last_change":"2023-03-03T06:25:13.842845+0000","last_active":"2023-03-03T06:25:18.344570+0000","last_peered":"2023-03-03T06:25:18.344570+0000","last_clean":"2023-03-03T06:25:18.344570+0000","last_became_active":"2023-03-03T06:25:13.842580+0000","last_became_peered":"2023-03-03T06:25:13.842580+0000","last_unstale":"2023-03-03T06:25:18.344570+0000","last_undegraded":"2023-03-03T06:25:18.344570+0000","last_fullsized":"2023-03-03T06:25:18.344570+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:28:42.514217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335348+0000","last_change":"2023-03-03T06:25:13.836306+0000","last_active":"2023-03-03T06:25:18.335348+0000","last_peered":"2023-03-03T06:25:18.335348+0000","last_clean":"2023-03-03T06:25:18.335348+0000","last_became_active":"2023-03-03T06:25:13.835953+0000","last_became_peered":"2023-03-03T06:25:13.835953+0000","last_unstale":"2023-03-03T06:25:18.335348+0000","last_undegraded":"2023-03-03T06:25:18.335348+0000","last_fullsized":"2023-03-03T06:25:18.335348+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:35:57.127264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.351349+0000","last_change":"2023-03-03T06:25:13.854132+0000","last_active":"2023-03-03T06:25:18.351349+0000","last_peered":"2023-03-03T06:25:18.351349+0000","last_clean":"2023-03-03T06:25:18.351349+0000","last_became_active":"2023-03-03T06:25:13.853786+0000","last_became_peered":"2023-03-03T06:25:13.853786+0000","last_unstale":"2023-03-03T06:25:18.351349+0000","last_undegraded":"2023-03-03T06:25:18.351349+0000","last_fullsized":"2023-03-03T06:25:18.351349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353001+0000","last_change":"2023-03-03T06:25:13.835134+0000","last_active":"2023-03-03T06:25:18.353001+0000","last_peered":"2023-03-03T06:25:18.353001+0000","last_clean":"2023-03-03T06:25:18.353001+0000","last_became_active":"2023-03-03T06:25:13.834844+0000","last_became_peered":"2023-03-03T06:25:13.834844+0000","last_unstale":"2023-03-03T06:25:18.353001+0000","last_undegraded":"2023-03-03T06:25:18.353001+0000","last_fullsized":"2023-03-03T06:25:18.353001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337727+0000","last_change":"2023-03-03T06:25:13.830031+0000","last_active":"2023-03-03T06:25:18.337727+0000","last_peered":"2023-03-03T06:25:18.337727+0000","last_clean":"2023-03-03T06:25:18.337727+0000","last_became_active":"2023-03-03T06:25:13.829770+0000","last_became_peered":"2023-03-03T06:25:13.829770+0000","last_unstale":"2023-03-03T06:25:18.337727+0000","last_undegraded":"2023-03-03T06:25:18.337727+0000","last_fullsized":"2023-03-03T06:25:18.337727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:06.900541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.336311+0000","last_change":"2023-03-03T06:25:13.858248+0000","last_active":"2023-03-03T06:25:18.336311+0000","last_peered":"2023-03-03T06:25:18.336311+0000","last_clean":"2023-03-03T06:25:18.336311+0000","last_became_active":"2023-03-03T06:25:13.857884+0000","last_became_peered":"2023-03-03T06:25:13.857884+0000","last_unstale":"2023-03-03T06:25:18.336311+0000","last_undegraded":"2023-03-03T06:25:18.336311+0000","last_fullsized":"2023-03-03T06:25:18.336311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:39.051115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334811+0000","last_change":"2023-03-03T06:25:13.847616+0000","last_active":"2023-03-03T06:25:18.334811+0000","last_peered":"2023-03-03T06:25:18.334811+0000","last_clean":"2023-03-03T06:25:18.334811+0000","last_became_active":"2023-03-03T06:25:13.847110+0000","last_became_peered":"2023-03-03T06:25:13.847110+0000","last_unstale":"2023-03-03T06:25:18.334811+0000","last_undegraded":"2023-03-03T06:25:18.334811+0000","last_fullsized":"2023-03-03T06:25:18.334811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:52:13.551129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353150+0000","last_change":"2023-03-03T06:25:13.851731+0000","last_active":"2023-03-03T06:25:18.353150+0000","last_peered":"2023-03-03T06:25:18.353150+0000","last_clean":"2023-03-03T06:25:18.353150+0000","last_became_active":"2023-03-03T06:25:13.849868+0000","last_became_peered":"2023-03-03T06:25:13.849868+0000","last_unstale":"2023-03-03T06:25:18.353150+0000","last_undegraded":"2023-03-03T06:25:18.353150+0000","last_fullsized":"2023-03-03T06:25:18.353150+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343321+0000","last_change":"2023-03-03T06:25:13.846247+0000","last_active":"2023-03-03T06:25:18.343321+0000","last_peered":"2023-03-03T06:25:18.343321+0000","last_clean":"2023-03-03T06:25:18.343321+0000","last_became_active":"2023-03-03T06:25:13.845983+0000","last_became_peered":"2023-03-03T06:25:13.845983+0000","last_unstale":"2023-03-03T06:25:18.343321+0000","last_undegraded":"2023-03-03T06:25:18.343321+0000","last_fullsized":"2023-03-03T06:25:18.343321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:54:02.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352766+0000","last_change":"2023-03-03T06:25:13.858013+0000","last_active":"2023-03-03T06:25:18.352766+0000","last_peered":"2023-03-03T06:25:18.352766+0000","last_clean":"2023-03-03T06:25:18.352766+0000","last_became_active":"2023-03-03T06:25:13.857706+0000","last_became_peered":"2023-03-03T06:25:13.857706+0000","last_unstale":"2023-03-03T06:25:18.352766+0000","last_undegraded":"2023-03-03T06:25:18.352766+0000","last_fullsized":"2023-03-03T06:25:18.352766+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:54:02.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.351182+0000","last_change":"2023-03-03T06:25:13.854691+0000","last_active":"2023-03-03T06:25:18.351182+0000","last_peered":"2023-03-03T06:25:18.351182+0000","last_clean":"2023-03-03T06:25:18.351182+0000","last_became_active":"2023-03-03T06:25:13.854332+0000","last_became_peered":"2023-03-03T06:25:13.854332+0000","last_unstale":"2023-03-03T06:25:18.351182+0000","last_undegraded":"2023-03-03T06:25:18.351182+0000","last_fullsized":"2023-03-03T06:25:18.351182+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334771+0000","last_change":"2023-03-03T06:25:13.836791+0000","last_active":"2023-03-03T06:25:18.334771+0000","last_peered":"2023-03-03T06:25:18.334771+0000","last_clean":"2023-03-03T06:25:18.334771+0000","last_became_active":"2023-03-03T06:25:13.836500+0000","last_became_peered":"2023-03-03T06:25:13.836500+0000","last_unstale":"2023-03-03T06:25:18.334771+0000","last_undegraded":"2023-03-03T06:25:18.334771+0000","last_fullsized":"2023-03-03T06:25:18.334771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:38:47.103593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344030+0000","last_change":"2023-03-03T06:25:13.846407+0000","last_active":"2023-03-03T06:25:18.344030+0000","last_peered":"2023-03-03T06:25:18.344030+0000","last_clean":"2023-03-03T06:25:18.344030+0000","last_became_active":"2023-03-03T06:25:13.846119+0000","last_became_peered":"2023-03-03T06:25:13.846119+0000","last_unstale":"2023-03-03T06:25:18.344030+0000","last_undegraded":"2023-03-03T06:25:18.344030+0000","last_fullsized":"2023-03-03T06:25:18.344030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341377+0000","last_change":"2023-03-03T06:25:13.843673+0000","last_active":"2023-03-03T06:25:18.341377+0000","last_peered":"2023-03-03T06:25:18.341377+0000","last_clean":"2023-03-03T06:25:18.341377+0000","last_became_active":"2023-03-03T06:25:13.843408+0000","last_became_peered":"2023-03-03T06:25:13.843408+0000","last_unstale":"2023-03-03T06:25:18.341377+0000","last_undegraded":"2023-03-03T06:25:18.341377+0000","last_fullsized":"2023-03-03T06:25:18.341377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:51:45.358951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335571+0000","last_change":"2023-03-03T06:25:13.847791+0000","last_active":"2023-03-03T06:25:18.335571+0000","last_peered":"2023-03-03T06:25:18.335571+0000","last_clean":"2023-03-03T06:25:18.335571+0000","last_became_active":"2023-03-03T06:25:13.847378+0000","last_became_peered":"2023-03-03T06:25:13.847378+0000","last_unstale":"2023-03-03T06:25:18.335571+0000","last_undegraded":"2023-03-03T06:25:18.335571+0000","last_fullsized":"2023-03-03T06:25:18.335571+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:05:53.754239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353098+0000","last_change":"2023-03-03T06:25:13.835727+0000","last_active":"2023-03-03T06:25:18.353098+0000","last_peered":"2023-03-03T06:25:18.353098+0000","last_clean":"2023-03-03T06:25:18.353098+0000","last_became_active":"2023-03-03T06:25:13.835462+0000","last_became_peered":"2023-03-03T06:25:13.835462+0000","last_unstale":"2023-03-03T06:25:18.353098+0000","last_undegraded":"2023-03-03T06:25:18.353098+0000","last_fullsized":"2023-03-03T06:25:18.353098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.336596+0000","last_change":"2023-03-03T06:25:13.827811+0000","last_active":"2023-03-03T06:25:18.336596+0000","last_peered":"2023-03-03T06:25:18.336596+0000","last_clean":"2023-03-03T06:25:18.336596+0000","last_became_active":"2023-03-03T06:25:13.827532+0000","last_became_peered":"2023-03-03T06:25:13.827532+0000","last_unstale":"2023-03-03T06:25:18.336596+0000","last_undegraded":"2023-03-03T06:25:18.336596+0000","last_fullsized":"2023-03-03T06:25:18.336596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:09.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4527,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13456,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":12544,"kb_avail":94358384,"statfs":{"total":96636764160,"available":96622985216,"internally_reserved":0,"allocated":925696,"data_stored":718901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12845056},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.639}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12272,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":11968,"kb_avail":94359568,"statfs":{"total":96636764160,"available":96624197632,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12255232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.772}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6519999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.577}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12280,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":11968,"kb_avail":94359560,"statfs":{"total":96636764160,"available":96624189440,"internally_reserved":0,"allocated":311296,"data_stored":125208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12255232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.609}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7989999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12356,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":12032,"kb_avail":94359484,"statfs":{"total":96636764160,"available":96624111616,"internally_reserved":0,"allocated":323584,"data_stored":128373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12320768},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.607}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6359999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8112,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363728,"statfs":{"total":96636764160,"available":96628457472,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7010000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8240,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363600,"statfs":{"total":96636764160,"available":96628326400,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.726}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.591}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.546}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9272,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362568,"statfs":{"total":96636764160,"available":96627269632,"internally_reserved":0,"allocated":901120,"data_stored":715506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.956}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8380000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-03T06:25:43.252 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-03T06:25:43.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.487+0000 7f373cb34700 1 Processor -- start 2023-03-03T06:25:43.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.487+0000 7f373cb34700 1 -- start start 2023-03-03T06:25:43.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.491+0000 7f373cb34700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f373815c750 0x7f3738158440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.491+0000 7f373cb34700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37380aee70 0x7f37380ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:43.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.491+0000 7f373cb34700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37380ca2a0 con 0x7f373806a780 2023-03-03T06:25:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.491+0000 7f373cb34700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37380cdd30 con 0x7f373815c750 2023-03-03T06:25:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.491+0000 7f373cb34700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3738158980 con 0x7f37380aee70 2023-03-03T06:25:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.491+0000 7f3736ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f373815c750 0x7f3738158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f3736ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f373815c750 0x7f3738158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43288/0 (socket says 172.21.15.3:43288) 2023-03-03T06:25:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f3736ffd700 1 -- 172.21.15.3:0/2017567816 learned_addr learned my addr 172.21.15.3:0/2017567816 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f37377fe700 1 --1- 172.21.15.3:0/2017567816 >> v1:172.21.15.3:6789/0 conn(0x7f373806a780 0x7f373815c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33632/0 (socket says 172.21.15.3:33632) 2023-03-03T06:25:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f3737fff700 1 --2- 172.21.15.3:0/2017567816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37380aee70 0x7f37380ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f3736ffd700 1 -- 172.21.15.3:0/2017567816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37380aee70 msgr2=0x7f37380ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f3736ffd700 1 --2- 172.21.15.3:0/2017567816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37380aee70 0x7f37380ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f3736ffd700 1 -- 172.21.15.3:0/2017567816 >> v1:172.21.15.3:6789/0 conn(0x7f373806a780 legacy=0x7f373815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f3736ffd700 1 -- 172.21.15.3:0/2017567816 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37380ca4d0 con 0x7f373815c750 2023-03-03T06:25:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f37367fc700 1 -- 172.21.15.3:0/2017567816 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588733599 0 0) 0x7f37380ca2a0 con 0x7f373806a780 2023-03-03T06:25:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f3736ffd700 1 --2- 172.21.15.3:0/2017567816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373815c750 0x7f3738158440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f372c009910 tx=0x7f372c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725b9a526e7490e5 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f37367fc700 1 -- 172.21.15.3:0/2017567816 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c011830 con 0x7f373815c750 2023-03-03T06:25:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f37367fc700 1 -- 172.21.15.3:0/2017567816 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c011990 con 0x7f373815c750 2023-03-03T06:25:43.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f37367fc700 1 -- 172.21.15.3:0/2017567816 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c011d40 con 0x7f373815c750 2023-03-03T06:25:43.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f373cb34700 1 -- 172.21.15.3:0/2017567816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373815c750 msgr2=0x7f3738158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:43.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.495+0000 7f373cb34700 1 --2- 172.21.15.3:0/2017567816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373815c750 0x7f3738158440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f372c009910 tx=0x7f372c009c20 comp rx=0 tx=0).stop 2023-03-03T06:25:43.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.499+0000 7f373cb34700 1 -- 172.21.15.3:0/2017567816 shutdown_connections 2023-03-03T06:25:43.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.499+0000 7f373cb34700 1 --2- 172.21.15.3:0/2017567816 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37380aee70 0x7f37380ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.499+0000 7f373cb34700 1 --2- 172.21.15.3:0/2017567816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373815c750 0x7f3738158440 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.499+0000 7f373cb34700 1 -- 172.21.15.3:0/2017567816 >> 172.21.15.3:0/2017567816 conn(0x7f3738070a30 msgr2=0x7f3738070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:43.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.499+0000 7f373cb34700 1 -- 172.21.15.3:0/2017567816 shutdown_connections 2023-03-03T06:25:43.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.499+0000 7f373cb34700 1 -- 172.21.15.3:0/2017567816 wait complete. 2023-03-03T06:25:43.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.499+0000 7f373cb34700 1 Processor -- start 2023-03-03T06:25:43.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f373cb34700 1 -- start start 2023-03-03T06:25:43.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f373cb34700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f373806a780 0x7f37381480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:43.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f373cb34700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37380aee70 0x7f37381f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f373cb34700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373815c750 0x7f373806c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f373cb34700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373806d5e0 con 0x7f373815c750 2023-03-03T06:25:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f373cb34700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37381f5c50 con 0x7f37380aee70 2023-03-03T06:25:43.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f373cb34700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37381f6be0 con 0x7f373806a780 2023-03-03T06:25:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f3737fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373815c750 0x7f373806c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f37377fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f373806a780 0x7f37381480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f37377fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f373806a780 0x7f37381480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:55242/0 (socket says 172.21.15.3:55242) 2023-03-03T06:25:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f37377fe700 1 -- 172.21.15.3:0/221096963 learned_addr learned my addr 172.21.15.3:0/221096963 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:43.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f3736ffd700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37380aee70 0x7f37381f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:43.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f3737fff700 1 -- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f373806a780 msgr2=0x7f37381480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:43.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f3737fff700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f373806a780 0x7f37381480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f3737fff700 1 -- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37380aee70 msgr2=0x7f37381f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:43.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f3737fff700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37380aee70 0x7f37381f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f3737fff700 1 -- 172.21.15.3:0/221096963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372c009570 con 0x7f373815c750 2023-03-03T06:25:43.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.503+0000 7f37377fe700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f373806a780 0x7f37381480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:43.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.507+0000 7f3737fff700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373815c750 0x7f373806c7d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f373000efc0 tx=0x7f373000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:43.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.507+0000 7f371ffff700 1 -- 172.21.15.3:0/221096963 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373000cc90 con 0x7f373815c750 2023-03-03T06:25:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.507+0000 7f371ffff700 1 -- 172.21.15.3:0/221096963 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373000f040 con 0x7f373815c750 2023-03-03T06:25:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.507+0000 7f371ffff700 1 -- 172.21.15.3:0/221096963 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37300146c0 con 0x7f373815c750 2023-03-03T06:25:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.507+0000 7f373cb34700 1 -- 172.21.15.3:0/221096963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37380cd9c0 con 0x7f373815c750 2023-03-03T06:25:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.507+0000 7f373cb34700 1 -- 172.21.15.3:0/221096963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37380cd6e0 con 0x7f373815c750 2023-03-03T06:25:43.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.507+0000 7f373cb34700 1 -- 172.21.15.3:0/221096963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37380ca7b0 con 0x7f373815c750 2023-03-03T06:25:43.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.511+0000 7f371ffff700 1 -- 172.21.15.3:0/221096963 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3730014820 con 0x7f373815c750 2023-03-03T06:25:43.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.511+0000 7f371ffff700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3724064ee0 0x7f3724067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:43.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.511+0000 7f371ffff700 1 -- 172.21.15.3:0/221096963 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3730082e90 con 0x7f373815c750 2023-03-03T06:25:43.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.515+0000 7f37377fe700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3724064ee0 0x7f3724067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:43.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.515+0000 7f37377fe700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3724064ee0 0x7f3724067390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3728000f60 tx=0x7f3728007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:43.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.519+0000 7f371ffff700 1 -- 172.21.15.3:0/221096963 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f3730050c50 con 0x7f373815c750 2023-03-03T06:25:43.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.691+0000 7f373cb34700 1 -- 172.21.15.3:0/221096963 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3738157610 con 0x7f3724064ee0 2023-03-03T06:25:43.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.711+0000 7f371ffff700 1 -- 172.21.15.3:0/221096963 <== mgr.4105 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367071 (secure 0 0 0) 0x7f3738157610 con 0x7f3724064ee0 2023-03-03T06:25:43.720 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:43.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.723+0000 7f373cb34700 1 -- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3724064ee0 msgr2=0x7f3724067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:43.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.723+0000 7f373cb34700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3724064ee0 0x7f3724067390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3728000f60 tx=0x7f3728007ec0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.723+0000 7f373cb34700 1 -- 172.21.15.3:0/221096963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373815c750 msgr2=0x7f373806c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:43.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.723+0000 7f373cb34700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373815c750 0x7f373806c7d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f373000efc0 tx=0x7f373000c3a0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.723+0000 7f373cb34700 1 -- 172.21.15.3:0/221096963 shutdown_connections 2023-03-03T06:25:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.723+0000 7f373cb34700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3724064ee0 0x7f3724067390 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.723+0000 7f373cb34700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f373815c750 0x7f373806c7d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.723+0000 7f373cb34700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37380aee70 0x7f37381f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.723+0000 7f373cb34700 1 --2- 172.21.15.3:0/221096963 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f373806a780 0x7f37381480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.723+0000 7f373cb34700 1 -- 172.21.15.3:0/221096963 >> 172.21.15.3:0/221096963 conn(0x7f3738070a30 msgr2=0x7f37380b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.723+0000 7f373cb34700 1 -- 172.21.15.3:0/221096963 shutdown_connections 2023-03-03T06:25:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:43.723+0000 7f373cb34700 1 -- 172.21.15.3:0/221096963 wait complete. 2023-03-03T06:25:43.732 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-03-03T06:25:43.764 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2023-03-03T06:25:42.408169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594895,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":84232,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":80576,"kb_avail":754890488,"statfs":{"total":773094113280,"available":773007859712,"internally_reserved":0,"allocated":3678208,"data_stored":2186366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":82509824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005964"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461806+0000","last_change":"2023-03-03T06:25:18.375133+0000","last_active":"2023-03-03T06:25:19.461806+0000","last_peered":"2023-03-03T06:25:19.461806+0000","last_clean":"2023-03-03T06:25:19.461806+0000","last_became_active":"2023-03-03T06:25:18.374500+0000","last_became_peered":"2023-03-03T06:25:18.374500+0000","last_unstale":"2023-03-03T06:25:19.461806+0000","last_undegraded":"2023-03-03T06:25:19.461806+0000","last_fullsized":"2023-03-03T06:25:19.461806+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:30.336080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343444+0000","last_change":"2023-03-03T06:25:13.301166+0000","last_active":"2023-03-03T06:25:18.343444+0000","last_peered":"2023-03-03T06:25:18.343444+0000","last_clean":"2023-03-03T06:25:18.343444+0000","last_became_active":"2023-03-03T06:25:13.300749+0000","last_became_peered":"2023-03-03T06:25:13.300749+0000","last_unstale":"2023-03-03T06:25:18.343444+0000","last_undegraded":"2023-03-03T06:25:18.343444+0000","last_fullsized":"2023-03-03T06:25:18.343444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335467+0000","last_change":"2023-03-03T06:25:13.839287+0000","last_active":"2023-03-03T06:25:18.335467+0000","last_peered":"2023-03-03T06:25:18.335467+0000","last_clean":"2023-03-03T06:25:18.335467+0000","last_became_active":"2023-03-03T06:25:13.838935+0000","last_became_peered":"2023-03-03T06:25:13.838935+0000","last_unstale":"2023-03-03T06:25:18.335467+0000","last_undegraded":"2023-03-03T06:25:18.335467+0000","last_fullsized":"2023-03-03T06:25:18.335467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:48:01.152785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.876352+0000","last_change":"2023-03-03T06:25:18.362828+0000","last_active":"2023-03-03T06:25:18.876352+0000","last_peered":"2023-03-03T06:25:18.876352+0000","last_clean":"2023-03-03T06:25:18.876352+0000","last_became_active":"2023-03-03T06:25:18.362423+0000","last_became_peered":"2023-03-03T06:25:18.362423+0000","last_unstale":"2023-03-03T06:25:18.876352+0000","last_undegraded":"2023-03-03T06:25:18.876352+0000","last_fullsized":"2023-03-03T06:25:18.876352+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:04:44.947509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.339522+0000","last_change":"2023-03-03T06:25:13.307540+0000","last_active":"2023-03-03T06:25:18.339522+0000","last_peered":"2023-03-03T06:25:18.339522+0000","last_clean":"2023-03-03T06:25:18.339522+0000","last_became_active":"2023-03-03T06:25:13.307229+0000","last_became_peered":"2023-03-03T06:25:13.307229+0000","last_unstale":"2023-03-03T06:25:18.339522+0000","last_undegraded":"2023-03-03T06:25:18.339522+0000","last_fullsized":"2023-03-03T06:25:18.339522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:09:34.703888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337997+0000","last_change":"2023-03-03T06:25:13.832118+0000","last_active":"2023-03-03T06:25:18.337997+0000","last_peered":"2023-03-03T06:25:18.337997+0000","last_clean":"2023-03-03T06:25:18.337997+0000","last_became_active":"2023-03-03T06:25:13.831744+0000","last_became_peered":"2023-03-03T06:25:13.831744+0000","last_unstale":"2023-03-03T06:25:18.337997+0000","last_undegraded":"2023-03-03T06:25:18.337997+0000","last_fullsized":"2023-03-03T06:25:18.337997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.878930+0000","last_change":"2023-03-03T06:25:18.367879+0000","last_active":"2023-03-03T06:25:18.878930+0000","last_peered":"2023-03-03T06:25:18.878930+0000","last_clean":"2023-03-03T06:25:18.878930+0000","last_became_active":"2023-03-03T06:25:18.367232+0000","last_became_peered":"2023-03-03T06:25:18.367232+0000","last_unstale":"2023-03-03T06:25:18.878930+0000","last_undegraded":"2023-03-03T06:25:18.878930+0000","last_fullsized":"2023-03-03T06:25:18.878930+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:43:20.174095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352877+0000","last_change":"2023-03-03T06:25:13.301087+0000","last_active":"2023-03-03T06:25:18.352877+0000","last_peered":"2023-03-03T06:25:18.352877+0000","last_clean":"2023-03-03T06:25:18.352877+0000","last_became_active":"2023-03-03T06:25:13.300837+0000","last_became_peered":"2023-03-03T06:25:13.300837+0000","last_unstale":"2023-03-03T06:25:18.352877+0000","last_undegraded":"2023-03-03T06:25:18.352877+0000","last_fullsized":"2023-03-03T06:25:18.352877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334901+0000","last_change":"2023-03-03T06:25:13.830954+0000","last_active":"2023-03-03T06:25:18.334901+0000","last_peered":"2023-03-03T06:25:18.334901+0000","last_clean":"2023-03-03T06:25:18.334901+0000","last_became_active":"2023-03-03T06:25:13.830614+0000","last_became_peered":"2023-03-03T06:25:13.830614+0000","last_unstale":"2023-03-03T06:25:18.334901+0000","last_undegraded":"2023-03-03T06:25:18.334901+0000","last_fullsized":"2023-03-03T06:25:18.334901+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.463051+0000","last_change":"2023-03-03T06:25:18.366781+0000","last_active":"2023-03-03T06:25:19.463051+0000","last_peered":"2023-03-03T06:25:19.463051+0000","last_clean":"2023-03-03T06:25:19.463051+0000","last_became_active":"2023-03-03T06:25:18.366175+0000","last_became_peered":"2023-03-03T06:25:18.366175+0000","last_unstale":"2023-03-03T06:25:19.463051+0000","last_undegraded":"2023-03-03T06:25:19.463051+0000","last_fullsized":"2023-03-03T06:25:19.463051+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:30.336080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352547+0000","last_change":"2023-03-03T06:25:13.300495+0000","last_active":"2023-03-03T06:25:18.352547+0000","last_peered":"2023-03-03T06:25:18.352547+0000","last_clean":"2023-03-03T06:25:18.352547+0000","last_became_active":"2023-03-03T06:25:13.300180+0000","last_became_peered":"2023-03-03T06:25:13.300180+0000","last_unstale":"2023-03-03T06:25:18.352547+0000","last_undegraded":"2023-03-03T06:25:18.352547+0000","last_fullsized":"2023-03-03T06:25:18.352547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335053+0000","last_change":"2023-03-03T06:25:13.830988+0000","last_active":"2023-03-03T06:25:18.335053+0000","last_peered":"2023-03-03T06:25:18.335053+0000","last_clean":"2023-03-03T06:25:18.335053+0000","last_became_active":"2023-03-03T06:25:13.826124+0000","last_became_peered":"2023-03-03T06:25:13.826124+0000","last_unstale":"2023-03-03T06:25:18.335053+0000","last_undegraded":"2023-03-03T06:25:18.335053+0000","last_fullsized":"2023-03-03T06:25:18.335053+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461936+0000","last_change":"2023-03-03T06:25:18.372519+0000","last_active":"2023-03-03T06:25:19.461936+0000","last_peered":"2023-03-03T06:25:19.461936+0000","last_clean":"2023-03-03T06:25:19.461936+0000","last_became_active":"2023-03-03T06:25:18.372087+0000","last_became_peered":"2023-03-03T06:25:18.372087+0000","last_unstale":"2023-03-03T06:25:19.461936+0000","last_undegraded":"2023-03-03T06:25:19.461936+0000","last_fullsized":"2023-03-03T06:25:19.461936+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:04.336725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344076+0000","last_change":"2023-03-03T06:25:13.308494+0000","last_active":"2023-03-03T06:25:18.344076+0000","last_peered":"2023-03-03T06:25:18.344076+0000","last_clean":"2023-03-03T06:25:18.344076+0000","last_became_active":"2023-03-03T06:25:13.308110+0000","last_became_peered":"2023-03-03T06:25:13.308110+0000","last_unstale":"2023-03-03T06:25:18.344076+0000","last_undegraded":"2023-03-03T06:25:18.344076+0000","last_fullsized":"2023-03-03T06:25:18.344076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.336445+0000","last_change":"2023-03-03T06:25:13.833412+0000","last_active":"2023-03-03T06:25:18.336445+0000","last_peered":"2023-03-03T06:25:18.336445+0000","last_clean":"2023-03-03T06:25:18.336445+0000","last_became_active":"2023-03-03T06:25:13.833138+0000","last_became_peered":"2023-03-03T06:25:13.833138+0000","last_unstale":"2023-03-03T06:25:18.336445+0000","last_undegraded":"2023-03-03T06:25:18.336445+0000","last_fullsized":"2023-03-03T06:25:18.336445+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:16:17.602774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.876849+0000","last_change":"2023-03-03T06:25:18.354570+0000","last_active":"2023-03-03T06:25:18.876849+0000","last_peered":"2023-03-03T06:25:18.876849+0000","last_clean":"2023-03-03T06:25:18.876849+0000","last_became_active":"2023-03-03T06:25:18.354128+0000","last_became_peered":"2023-03-03T06:25:18.354128+0000","last_unstale":"2023-03-03T06:25:18.876849+0000","last_undegraded":"2023-03-03T06:25:18.876849+0000","last_fullsized":"2023-03-03T06:25:18.876849+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:43:00.515957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335127+0000","last_change":"2023-03-03T06:25:13.307789+0000","last_active":"2023-03-03T06:25:18.335127+0000","last_peered":"2023-03-03T06:25:18.335127+0000","last_clean":"2023-03-03T06:25:18.335127+0000","last_became_active":"2023-03-03T06:25:13.307478+0000","last_became_peered":"2023-03-03T06:25:13.307478+0000","last_unstale":"2023-03-03T06:25:18.335127+0000","last_undegraded":"2023-03-03T06:25:18.335127+0000","last_fullsized":"2023-03-03T06:25:18.335127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:11:32.033289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.339285+0000","last_change":"2023-03-03T06:25:13.830797+0000","last_active":"2023-03-03T06:25:18.339285+0000","last_peered":"2023-03-03T06:25:18.339285+0000","last_clean":"2023-03-03T06:25:18.339285+0000","last_became_active":"2023-03-03T06:25:13.829482+0000","last_became_peered":"2023-03-03T06:25:13.829482+0000","last_unstale":"2023-03-03T06:25:18.339285+0000","last_undegraded":"2023-03-03T06:25:18.339285+0000","last_fullsized":"2023-03-03T06:25:18.339285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:48:01.152785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.459250+0000","last_change":"2023-03-03T06:25:18.358329+0000","last_active":"2023-03-03T06:25:19.459250+0000","last_peered":"2023-03-03T06:25:19.459250+0000","last_clean":"2023-03-03T06:25:19.459250+0000","last_became_active":"2023-03-03T06:25:18.357641+0000","last_became_peered":"2023-03-03T06:25:18.357641+0000","last_unstale":"2023-03-03T06:25:19.459250+0000","last_undegraded":"2023-03-03T06:25:19.459250+0000","last_fullsized":"2023-03-03T06:25:19.459250+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:59.071940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.359353+0000","last_change":"2023-03-03T06:25:13.312665+0000","last_active":"2023-03-03T06:25:18.359353+0000","last_peered":"2023-03-03T06:25:18.359353+0000","last_clean":"2023-03-03T06:25:18.359353+0000","last_became_active":"2023-03-03T06:25:13.312255+0000","last_became_peered":"2023-03-03T06:25:13.312255+0000","last_unstale":"2023-03-03T06:25:18.359353+0000","last_undegraded":"2023-03-03T06:25:18.359353+0000","last_fullsized":"2023-03-03T06:25:18.359353+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344419+0000","last_change":"2023-03-03T06:25:13.847886+0000","last_active":"2023-03-03T06:25:18.344419+0000","last_peered":"2023-03-03T06:25:18.344419+0000","last_clean":"2023-03-03T06:25:18.344419+0000","last_became_active":"2023-03-03T06:25:13.847620+0000","last_became_peered":"2023-03-03T06:25:13.847620+0000","last_unstale":"2023-03-03T06:25:18.344419+0000","last_undegraded":"2023-03-03T06:25:18.344419+0000","last_fullsized":"2023-03-03T06:25:18.344419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:48:01.152785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.875799+0000","last_change":"2023-03-03T06:25:18.374567+0000","last_active":"2023-03-03T06:25:18.875799+0000","last_peered":"2023-03-03T06:25:18.875799+0000","last_clean":"2023-03-03T06:25:18.875799+0000","last_became_active":"2023-03-03T06:25:18.374172+0000","last_became_peered":"2023-03-03T06:25:18.374172+0000","last_unstale":"2023-03-03T06:25:18.875799+0000","last_undegraded":"2023-03-03T06:25:18.875799+0000","last_fullsized":"2023-03-03T06:25:18.875799+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:11:32.109566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341581+0000","last_change":"2023-03-03T06:25:13.310405+0000","last_active":"2023-03-03T06:25:18.341581+0000","last_peered":"2023-03-03T06:25:18.341581+0000","last_clean":"2023-03-03T06:25:18.341581+0000","last_became_active":"2023-03-03T06:25:13.310029+0000","last_became_peered":"2023-03-03T06:25:13.310029+0000","last_unstale":"2023-03-03T06:25:18.341581+0000","last_undegraded":"2023-03-03T06:25:18.341581+0000","last_fullsized":"2023-03-03T06:25:18.341581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:16:16.586666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343606+0000","last_change":"2023-03-03T06:25:13.839392+0000","last_active":"2023-03-03T06:25:18.343606+0000","last_peered":"2023-03-03T06:25:18.343606+0000","last_clean":"2023-03-03T06:25:18.343606+0000","last_became_active":"2023-03-03T06:25:13.839078+0000","last_became_peered":"2023-03-03T06:25:13.839078+0000","last_unstale":"2023-03-03T06:25:18.343606+0000","last_undegraded":"2023-03-03T06:25:18.343606+0000","last_fullsized":"2023-03-03T06:25:18.343606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:42:30.992559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341473+0000","last_change":"2023-03-03T06:25:13.836064+0000","last_active":"2023-03-03T06:25:18.341473+0000","last_peered":"2023-03-03T06:25:18.341473+0000","last_clean":"2023-03-03T06:25:18.341473+0000","last_became_active":"2023-03-03T06:25:13.835743+0000","last_became_peered":"2023-03-03T06:25:13.835743+0000","last_unstale":"2023-03-03T06:25:18.341473+0000","last_undegraded":"2023-03-03T06:25:18.341473+0000","last_fullsized":"2023-03-03T06:25:18.341473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:52:13.551129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343523+0000","last_change":"2023-03-03T06:25:13.293588+0000","last_active":"2023-03-03T06:25:18.343523+0000","last_peered":"2023-03-03T06:25:18.343523+0000","last_clean":"2023-03-03T06:25:18.343523+0000","last_became_active":"2023-03-03T06:25:13.292998+0000","last_became_peered":"2023-03-03T06:25:13.292998+0000","last_unstale":"2023-03-03T06:25:18.343523+0000","last_undegraded":"2023-03-03T06:25:18.343523+0000","last_fullsized":"2023-03-03T06:25:18.343523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:17:34.388274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.459022+0000","last_change":"2023-03-03T06:25:18.365923+0000","last_active":"2023-03-03T06:25:19.459022+0000","last_peered":"2023-03-03T06:25:19.459022+0000","last_clean":"2023-03-03T06:25:19.459022+0000","last_became_active":"2023-03-03T06:25:18.365397+0000","last_became_peered":"2023-03-03T06:25:18.365397+0000","last_unstale":"2023-03-03T06:25:19.459022+0000","last_undegraded":"2023-03-03T06:25:19.459022+0000","last_fullsized":"2023-03-03T06:25:19.459022+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:45:30.096214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337824+0000","last_change":"2023-03-03T06:25:13.837989+0000","last_active":"2023-03-03T06:25:18.337824+0000","last_peered":"2023-03-03T06:25:18.337824+0000","last_clean":"2023-03-03T06:25:18.337824+0000","last_became_active":"2023-03-03T06:25:13.837724+0000","last_became_peered":"2023-03-03T06:25:13.837724+0000","last_unstale":"2023-03-03T06:25:18.337824+0000","last_undegraded":"2023-03-03T06:25:18.337824+0000","last_fullsized":"2023-03-03T06:25:18.337824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353186+0000","last_change":"2023-03-03T06:25:13.302153+0000","last_active":"2023-03-03T06:25:18.353186+0000","last_peered":"2023-03-03T06:25:18.353186+0000","last_clean":"2023-03-03T06:25:18.353186+0000","last_became_active":"2023-03-03T06:25:13.301751+0000","last_became_peered":"2023-03-03T06:25:13.301751+0000","last_unstale":"2023-03-03T06:25:18.353186+0000","last_undegraded":"2023-03-03T06:25:18.353186+0000","last_fullsized":"2023-03-03T06:25:18.353186+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.462897+0000","last_change":"2023-03-03T06:25:18.366010+0000","last_active":"2023-03-03T06:25:19.462897+0000","last_peered":"2023-03-03T06:25:19.462897+0000","last_clean":"2023-03-03T06:25:19.462897+0000","last_became_active":"2023-03-03T06:25:18.362457+0000","last_became_peered":"2023-03-03T06:25:18.362457+0000","last_unstale":"2023-03-03T06:25:19.462897+0000","last_undegraded":"2023-03-03T06:25:19.462897+0000","last_fullsized":"2023-03-03T06:25:19.462897+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:17:02.442524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350698+0000","last_change":"2023-03-03T06:25:13.312520+0000","last_active":"2023-03-03T06:25:18.350698+0000","last_peered":"2023-03-03T06:25:18.350698+0000","last_clean":"2023-03-03T06:25:18.350698+0000","last_became_active":"2023-03-03T06:25:13.312105+0000","last_became_peered":"2023-03-03T06:25:13.312105+0000","last_unstale":"2023-03-03T06:25:18.350698+0000","last_undegraded":"2023-03-03T06:25:18.350698+0000","last_fullsized":"2023-03-03T06:25:18.350698+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:51:29.967970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358702+0000","last_change":"2023-03-03T06:25:13.835492+0000","last_active":"2023-03-03T06:25:18.358702+0000","last_peered":"2023-03-03T06:25:18.358702+0000","last_clean":"2023-03-03T06:25:18.358702+0000","last_became_active":"2023-03-03T06:25:13.835173+0000","last_became_peered":"2023-03-03T06:25:13.835173+0000","last_unstale":"2023-03-03T06:25:18.358702+0000","last_undegraded":"2023-03-03T06:25:18.358702+0000","last_fullsized":"2023-03-03T06:25:18.358702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467726+0000","last_change":"2023-03-03T06:25:18.362177+0000","last_active":"2023-03-03T06:25:19.467726+0000","last_peered":"2023-03-03T06:25:19.467726+0000","last_clean":"2023-03-03T06:25:19.467726+0000","last_became_active":"2023-03-03T06:25:18.361833+0000","last_became_peered":"2023-03-03T06:25:18.361833+0000","last_unstale":"2023-03-03T06:25:19.467726+0000","last_undegraded":"2023-03-03T06:25:19.467726+0000","last_fullsized":"2023-03-03T06:25:19.467726+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:04:44.947509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:32.598145+0000","last_change":"2023-03-03T06:25:13.302558+0000","last_active":"2023-03-03T06:25:32.598145+0000","last_peered":"2023-03-03T06:25:32.598145+0000","last_clean":"2023-03-03T06:25:32.598145+0000","last_became_active":"2023-03-03T06:25:13.301983+0000","last_became_peered":"2023-03-03T06:25:13.301983+0000","last_unstale":"2023-03-03T06:25:32.598145+0000","last_undegraded":"2023-03-03T06:25:32.598145+0000","last_fullsized":"2023-03-03T06:25:32.598145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:34:41.158814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3745,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334685+0000","last_change":"2023-03-03T06:25:13.835359+0000","last_active":"2023-03-03T06:25:18.334685+0000","last_peered":"2023-03-03T06:25:18.334685+0000","last_clean":"2023-03-03T06:25:18.334685+0000","last_became_active":"2023-03-03T06:25:13.835103+0000","last_became_peered":"2023-03-03T06:25:13.835103+0000","last_unstale":"2023-03-03T06:25:18.334685+0000","last_undegraded":"2023-03-03T06:25:18.334685+0000","last_fullsized":"2023-03-03T06:25:18.334685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461833+0000","last_change":"2023-03-03T06:25:18.376174+0000","last_active":"2023-03-03T06:25:19.461833+0000","last_peered":"2023-03-03T06:25:19.461833+0000","last_clean":"2023-03-03T06:25:19.461833+0000","last_became_active":"2023-03-03T06:25:18.375779+0000","last_became_peered":"2023-03-03T06:25:18.375779+0000","last_unstale":"2023-03-03T06:25:19.461833+0000","last_undegraded":"2023-03-03T06:25:19.461833+0000","last_fullsized":"2023-03-03T06:25:19.461833+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:40:49.542836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335280+0000","last_change":"2023-03-03T06:25:13.837277+0000","last_active":"2023-03-03T06:25:18.335280+0000","last_peered":"2023-03-03T06:25:18.335280+0000","last_clean":"2023-03-03T06:25:18.335280+0000","last_became_active":"2023-03-03T06:25:13.836510+0000","last_became_peered":"2023-03-03T06:25:13.836510+0000","last_unstale":"2023-03-03T06:25:18.335280+0000","last_undegraded":"2023-03-03T06:25:18.335280+0000","last_fullsized":"2023-03-03T06:25:18.335280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343682+0000","last_change":"2023-03-03T06:25:07.205700+0000","last_active":"2023-03-03T06:25:18.343682+0000","last_peered":"2023-03-03T06:25:18.343682+0000","last_clean":"2023-03-03T06:25:18.343682+0000","last_became_active":"2023-03-03T06:25:07.205269+0000","last_became_peered":"2023-03-03T06:25:07.205269+0000","last_unstale":"2023-03-03T06:25:18.343682+0000","last_undegraded":"2023-03-03T06:25:18.343682+0000","last_fullsized":"2023-03-03T06:25:18.343682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:17:53.442570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344712+0000","last_change":"2023-03-03T06:25:13.309537+0000","last_active":"2023-03-03T06:25:18.344712+0000","last_peered":"2023-03-03T06:25:18.344712+0000","last_clean":"2023-03-03T06:25:18.344712+0000","last_became_active":"2023-03-03T06:25:13.309208+0000","last_became_peered":"2023-03-03T06:25:13.309208+0000","last_unstale":"2023-03-03T06:25:18.344712+0000","last_undegraded":"2023-03-03T06:25:18.344712+0000","last_fullsized":"2023-03-03T06:25:18.344712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:11:45.788884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467622+0000","last_change":"2023-03-03T06:25:18.367070+0000","last_active":"2023-03-03T06:25:19.467622+0000","last_peered":"2023-03-03T06:25:19.467622+0000","last_clean":"2023-03-03T06:25:19.467622+0000","last_became_active":"2023-03-03T06:25:18.366674+0000","last_became_peered":"2023-03-03T06:25:18.366674+0000","last_unstale":"2023-03-03T06:25:19.467622+0000","last_undegraded":"2023-03-03T06:25:19.467622+0000","last_fullsized":"2023-03-03T06:25:19.467622+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:43:20.174095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341556+0000","last_change":"2023-03-03T06:25:13.837256+0000","last_active":"2023-03-03T06:25:18.341556+0000","last_peered":"2023-03-03T06:25:18.341556+0000","last_clean":"2023-03-03T06:25:18.341556+0000","last_became_active":"2023-03-03T06:25:13.836942+0000","last_became_peered":"2023-03-03T06:25:13.836942+0000","last_unstale":"2023-03-03T06:25:18.341556+0000","last_undegraded":"2023-03-03T06:25:18.341556+0000","last_fullsized":"2023-03-03T06:25:18.341556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:05:53.754239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357705+0000","last_change":"2023-03-03T06:25:07.204903+0000","last_active":"2023-03-03T06:25:18.357705+0000","last_peered":"2023-03-03T06:25:18.357705+0000","last_clean":"2023-03-03T06:25:18.357705+0000","last_became_active":"2023-03-03T06:25:07.204601+0000","last_became_peered":"2023-03-03T06:25:07.204601+0000","last_unstale":"2023-03-03T06:25:18.357705+0000","last_undegraded":"2023-03-03T06:25:18.357705+0000","last_fullsized":"2023-03-03T06:25:18.357705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:18:36.947912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343364+0000","last_change":"2023-03-03T06:25:13.304543+0000","last_active":"2023-03-03T06:25:18.343364+0000","last_peered":"2023-03-03T06:25:18.343364+0000","last_clean":"2023-03-03T06:25:18.343364+0000","last_became_active":"2023-03-03T06:25:13.304142+0000","last_became_peered":"2023-03-03T06:25:13.304142+0000","last_unstale":"2023-03-03T06:25:18.343364+0000","last_undegraded":"2023-03-03T06:25:18.343364+0000","last_fullsized":"2023-03-03T06:25:18.343364+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.876556+0000","last_change":"2023-03-03T06:25:18.354139+0000","last_active":"2023-03-03T06:25:18.876556+0000","last_peered":"2023-03-03T06:25:18.876556+0000","last_clean":"2023-03-03T06:25:18.876556+0000","last_became_active":"2023-03-03T06:25:18.353543+0000","last_became_peered":"2023-03-03T06:25:18.353543+0000","last_unstale":"2023-03-03T06:25:18.876556+0000","last_undegraded":"2023-03-03T06:25:18.876556+0000","last_fullsized":"2023-03-03T06:25:18.876556+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:11:23.940316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343906+0000","last_change":"2023-03-03T06:25:13.838819+0000","last_active":"2023-03-03T06:25:18.343906+0000","last_peered":"2023-03-03T06:25:18.343906+0000","last_clean":"2023-03-03T06:25:18.343906+0000","last_became_active":"2023-03-03T06:25:13.838558+0000","last_became_peered":"2023-03-03T06:25:13.838558+0000","last_unstale":"2023-03-03T06:25:18.343906+0000","last_undegraded":"2023-03-03T06:25:18.343906+0000","last_fullsized":"2023-03-03T06:25:18.343906+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350807+0000","last_change":"2023-03-03T06:25:07.208473+0000","last_active":"2023-03-03T06:25:18.350807+0000","last_peered":"2023-03-03T06:25:18.350807+0000","last_clean":"2023-03-03T06:25:18.350807+0000","last_became_active":"2023-03-03T06:25:07.208041+0000","last_became_peered":"2023-03-03T06:25:07.208041+0000","last_unstale":"2023-03-03T06:25:18.350807+0000","last_undegraded":"2023-03-03T06:25:18.350807+0000","last_fullsized":"2023-03-03T06:25:18.350807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:22:19.852493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357626+0000","last_change":"2023-03-03T06:25:13.308259+0000","last_active":"2023-03-03T06:25:18.357626+0000","last_peered":"2023-03-03T06:25:18.357626+0000","last_clean":"2023-03-03T06:25:18.357626+0000","last_became_active":"2023-03-03T06:25:13.307902+0000","last_became_peered":"2023-03-03T06:25:13.307902+0000","last_unstale":"2023-03-03T06:25:18.357626+0000","last_undegraded":"2023-03-03T06:25:18.357626+0000","last_fullsized":"2023-03-03T06:25:18.357626+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.459074+0000","last_change":"2023-03-03T06:25:18.355878+0000","last_active":"2023-03-03T06:25:19.459074+0000","last_peered":"2023-03-03T06:25:19.459074+0000","last_clean":"2023-03-03T06:25:19.459074+0000","last_became_active":"2023-03-03T06:25:18.355460+0000","last_became_peered":"2023-03-03T06:25:18.355460+0000","last_unstale":"2023-03-03T06:25:19.459074+0000","last_undegraded":"2023-03-03T06:25:19.459074+0000","last_fullsized":"2023-03-03T06:25:19.459074+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:38:51.150150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358771+0000","last_change":"2023-03-03T06:25:13.839947+0000","last_active":"2023-03-03T06:25:18.358771+0000","last_peered":"2023-03-03T06:25:18.358771+0000","last_clean":"2023-03-03T06:25:18.358771+0000","last_became_active":"2023-03-03T06:25:13.839632+0000","last_became_peered":"2023-03-03T06:25:13.839632+0000","last_unstale":"2023-03-03T06:25:18.358771+0000","last_undegraded":"2023-03-03T06:25:18.358771+0000","last_fullsized":"2023-03-03T06:25:18.358771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357780+0000","last_change":"2023-03-03T06:25:07.201865+0000","last_active":"2023-03-03T06:25:18.357780+0000","last_peered":"2023-03-03T06:25:18.357780+0000","last_clean":"2023-03-03T06:25:18.357780+0000","last_became_active":"2023-03-03T06:25:07.201411+0000","last_became_peered":"2023-03-03T06:25:07.201411+0000","last_unstale":"2023-03-03T06:25:18.357780+0000","last_undegraded":"2023-03-03T06:25:18.357780+0000","last_fullsized":"2023-03-03T06:25:18.357780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:17:53.442570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352970+0000","last_change":"2023-03-03T06:25:13.312426+0000","last_active":"2023-03-03T06:25:18.352970+0000","last_peered":"2023-03-03T06:25:18.352970+0000","last_clean":"2023-03-03T06:25:18.352970+0000","last_became_active":"2023-03-03T06:25:13.312129+0000","last_became_peered":"2023-03-03T06:25:13.312129+0000","last_unstale":"2023-03-03T06:25:18.352970+0000","last_undegraded":"2023-03-03T06:25:18.352970+0000","last_fullsized":"2023-03-03T06:25:18.352970+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.875532+0000","last_change":"2023-03-03T06:25:18.365085+0000","last_active":"2023-03-03T06:25:18.875532+0000","last_peered":"2023-03-03T06:25:18.875532+0000","last_clean":"2023-03-03T06:25:18.875532+0000","last_became_active":"2023-03-03T06:25:18.364690+0000","last_became_peered":"2023-03-03T06:25:18.364690+0000","last_unstale":"2023-03-03T06:25:18.875532+0000","last_undegraded":"2023-03-03T06:25:18.875532+0000","last_fullsized":"2023-03-03T06:25:18.875532+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:30.336080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357548+0000","last_change":"2023-03-03T06:25:13.828855+0000","last_active":"2023-03-03T06:25:18.357548+0000","last_peered":"2023-03-03T06:25:18.357548+0000","last_clean":"2023-03-03T06:25:18.357548+0000","last_became_active":"2023-03-03T06:25:13.828546+0000","last_became_peered":"2023-03-03T06:25:13.828546+0000","last_unstale":"2023-03-03T06:25:18.357548+0000","last_undegraded":"2023-03-03T06:25:18.357548+0000","last_fullsized":"2023-03-03T06:25:18.357548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334824+0000","last_change":"2023-03-03T06:25:13.298859+0000","last_active":"2023-03-03T06:25:18.334824+0000","last_peered":"2023-03-03T06:25:18.334824+0000","last_clean":"2023-03-03T06:25:18.334824+0000","last_became_active":"2023-03-03T06:25:13.298586+0000","last_became_peered":"2023-03-03T06:25:13.298586+0000","last_unstale":"2023-03-03T06:25:18.334824+0000","last_undegraded":"2023-03-03T06:25:18.334824+0000","last_fullsized":"2023-03-03T06:25:18.334824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:11:45.788884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353089+0000","last_change":"2023-03-03T06:25:07.201941+0000","last_active":"2023-03-03T06:25:18.353089+0000","last_peered":"2023-03-03T06:25:18.353089+0000","last_clean":"2023-03-03T06:25:18.353089+0000","last_became_active":"2023-03-03T06:25:07.201437+0000","last_became_peered":"2023-03-03T06:25:07.201437+0000","last_unstale":"2023-03-03T06:25:18.353089+0000","last_undegraded":"2023-03-03T06:25:18.353089+0000","last_fullsized":"2023-03-03T06:25:18.353089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:48:52.145810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.875719+0000","last_change":"2023-03-03T06:25:18.374636+0000","last_active":"2023-03-03T06:25:18.875719+0000","last_peered":"2023-03-03T06:25:18.875719+0000","last_clean":"2023-03-03T06:25:18.875719+0000","last_became_active":"2023-03-03T06:25:18.373925+0000","last_became_peered":"2023-03-03T06:25:18.373925+0000","last_unstale":"2023-03-03T06:25:18.875719+0000","last_undegraded":"2023-03-03T06:25:18.875719+0000","last_fullsized":"2023-03-03T06:25:18.875719+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:07:38.561763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.359409+0000","last_change":"2023-03-03T06:25:13.833104+0000","last_active":"2023-03-03T06:25:18.359409+0000","last_peered":"2023-03-03T06:25:18.359409+0000","last_clean":"2023-03-03T06:25:18.359409+0000","last_became_active":"2023-03-03T06:25:13.832815+0000","last_became_peered":"2023-03-03T06:25:13.832815+0000","last_unstale":"2023-03-03T06:25:18.359409+0000","last_undegraded":"2023-03-03T06:25:18.359409+0000","last_fullsized":"2023-03-03T06:25:18.359409+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.356972+0000","last_change":"2023-03-03T06:25:13.298339+0000","last_active":"2023-03-03T06:25:18.356972+0000","last_peered":"2023-03-03T06:25:18.356972+0000","last_clean":"2023-03-03T06:25:18.356972+0000","last_became_active":"2023-03-03T06:25:13.297116+0000","last_became_peered":"2023-03-03T06:25:13.297116+0000","last_unstale":"2023-03-03T06:25:18.356972+0000","last_undegraded":"2023-03-03T06:25:18.356972+0000","last_fullsized":"2023-03-03T06:25:18.356972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335947+0000","last_change":"2023-03-03T06:25:07.437275+0000","last_active":"2023-03-03T06:25:18.335947+0000","last_peered":"2023-03-03T06:25:18.335947+0000","last_clean":"2023-03-03T06:25:18.335947+0000","last_became_active":"2023-03-03T06:25:07.436823+0000","last_became_peered":"2023-03-03T06:25:07.436823+0000","last_unstale":"2023-03-03T06:25:18.335947+0000","last_undegraded":"2023-03-03T06:25:18.335947+0000","last_fullsized":"2023-03-03T06:25:18.335947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:24:15.529041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217928+0000","last_change":"2023-03-03T06:25:18.361949+0000","last_active":"2023-03-03T06:25:19.217928+0000","last_peered":"2023-03-03T06:25:19.217928+0000","last_clean":"2023-03-03T06:25:19.217928+0000","last_became_active":"2023-03-03T06:25:18.361407+0000","last_became_peered":"2023-03-03T06:25:18.361407+0000","last_unstale":"2023-03-03T06:25:19.217928+0000","last_undegraded":"2023-03-03T06:25:19.217928+0000","last_fullsized":"2023-03-03T06:25:19.217928+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:59.071940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344511+0000","last_change":"2023-03-03T06:25:13.833788+0000","last_active":"2023-03-03T06:25:18.344511+0000","last_peered":"2023-03-03T06:25:18.344511+0000","last_clean":"2023-03-03T06:25:18.344511+0000","last_became_active":"2023-03-03T06:25:13.833532+0000","last_became_peered":"2023-03-03T06:25:13.833532+0000","last_unstale":"2023-03-03T06:25:18.344511+0000","last_undegraded":"2023-03-03T06:25:18.344511+0000","last_fullsized":"2023-03-03T06:25:18.344511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:59:50.232210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342831+0000","last_change":"2023-03-03T06:25:13.308357+0000","last_active":"2023-03-03T06:25:18.342831+0000","last_peered":"2023-03-03T06:25:18.342831+0000","last_clean":"2023-03-03T06:25:18.342831+0000","last_became_active":"2023-03-03T06:25:13.307981+0000","last_became_peered":"2023-03-03T06:25:13.307981+0000","last_unstale":"2023-03-03T06:25:18.342831+0000","last_undegraded":"2023-03-03T06:25:18.342831+0000","last_fullsized":"2023-03-03T06:25:18.342831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346873+0000","last_change":"2023-03-03T06:25:07.208585+0000","last_active":"2023-03-03T06:25:18.346873+0000","last_peered":"2023-03-03T06:25:18.346873+0000","last_clean":"2023-03-03T06:25:18.346873+0000","last_became_active":"2023-03-03T06:25:07.208257+0000","last_became_peered":"2023-03-03T06:25:07.208257+0000","last_unstale":"2023-03-03T06:25:18.346873+0000","last_undegraded":"2023-03-03T06:25:18.346873+0000","last_fullsized":"2023-03-03T06:25:18.346873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:11:52.893733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879358+0000","last_change":"2023-03-03T06:25:18.369960+0000","last_active":"2023-03-03T06:25:18.879358+0000","last_peered":"2023-03-03T06:25:18.879358+0000","last_clean":"2023-03-03T06:25:18.879358+0000","last_became_active":"2023-03-03T06:25:18.367846+0000","last_became_peered":"2023-03-03T06:25:18.367846+0000","last_unstale":"2023-03-03T06:25:18.879358+0000","last_undegraded":"2023-03-03T06:25:18.879358+0000","last_fullsized":"2023-03-03T06:25:18.879358+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:12:09.571057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342233+0000","last_change":"2023-03-03T06:25:13.857729+0000","last_active":"2023-03-03T06:25:18.342233+0000","last_peered":"2023-03-03T06:25:18.342233+0000","last_clean":"2023-03-03T06:25:18.342233+0000","last_became_active":"2023-03-03T06:25:13.857453+0000","last_became_peered":"2023-03-03T06:25:13.857453+0000","last_unstale":"2023-03-03T06:25:18.342233+0000","last_undegraded":"2023-03-03T06:25:18.342233+0000","last_fullsized":"2023-03-03T06:25:18.342233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:46:08.313482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.339103+0000","last_change":"2023-03-03T06:25:13.303419+0000","last_active":"2023-03-03T06:25:18.339103+0000","last_peered":"2023-03-03T06:25:18.339103+0000","last_clean":"2023-03-03T06:25:18.339103+0000","last_became_active":"2023-03-03T06:25:13.303084+0000","last_became_peered":"2023-03-03T06:25:13.303084+0000","last_unstale":"2023-03-03T06:25:18.339103+0000","last_undegraded":"2023-03-03T06:25:18.339103+0000","last_fullsized":"2023-03-03T06:25:18.339103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352430+0000","last_change":"2023-03-03T06:25:07.203115+0000","last_active":"2023-03-03T06:25:18.352430+0000","last_peered":"2023-03-03T06:25:18.352430+0000","last_clean":"2023-03-03T06:25:18.352430+0000","last_became_active":"2023-03-03T06:25:07.202705+0000","last_became_peered":"2023-03-03T06:25:07.202705+0000","last_unstale":"2023-03-03T06:25:18.352430+0000","last_undegraded":"2023-03-03T06:25:18.352430+0000","last_fullsized":"2023-03-03T06:25:18.352430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:11:32.938698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346951+0000","last_change":"2023-03-03T06:25:02.138688+0000","last_active":"2023-03-03T06:25:18.346951+0000","last_peered":"2023-03-03T06:25:18.346951+0000","last_clean":"2023-03-03T06:25:18.346951+0000","last_became_active":"2023-03-03T06:25:02.137945+0000","last_became_peered":"2023-03-03T06:25:02.137945+0000","last_unstale":"2023-03-03T06:25:18.346951+0000","last_undegraded":"2023-03-03T06:25:18.346951+0000","last_fullsized":"2023-03-03T06:25:18.346951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:00.679233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:00.679233+0000","last_clean_scrub_stamp":"2023-03-03T06:25:00.679233+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:24:46.304379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.463093+0000","last_change":"2023-03-03T06:25:18.368371+0000","last_active":"2023-03-03T06:25:19.463093+0000","last_peered":"2023-03-03T06:25:19.463093+0000","last_clean":"2023-03-03T06:25:19.463093+0000","last_became_active":"2023-03-03T06:25:18.367896+0000","last_became_peered":"2023-03-03T06:25:18.367896+0000","last_unstale":"2023-03-03T06:25:19.463093+0000","last_undegraded":"2023-03-03T06:25:19.463093+0000","last_fullsized":"2023-03-03T06:25:19.463093+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:43:20.174095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358378+0000","last_change":"2023-03-03T06:25:13.842311+0000","last_active":"2023-03-03T06:25:18.358378+0000","last_peered":"2023-03-03T06:25:18.358378+0000","last_clean":"2023-03-03T06:25:18.358378+0000","last_became_active":"2023-03-03T06:25:13.841952+0000","last_became_peered":"2023-03-03T06:25:13.841952+0000","last_unstale":"2023-03-03T06:25:18.358378+0000","last_undegraded":"2023-03-03T06:25:18.358378+0000","last_fullsized":"2023-03-03T06:25:18.358378+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:46:08.313482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341679+0000","last_change":"2023-03-03T06:25:13.306584+0000","last_active":"2023-03-03T06:25:18.341679+0000","last_peered":"2023-03-03T06:25:18.341679+0000","last_clean":"2023-03-03T06:25:18.341679+0000","last_became_active":"2023-03-03T06:25:13.306295+0000","last_became_peered":"2023-03-03T06:25:13.306295+0000","last_unstale":"2023-03-03T06:25:18.341679+0000","last_undegraded":"2023-03-03T06:25:18.341679+0000","last_fullsized":"2023-03-03T06:25:18.341679+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.878155+0000","last_change":"2023-03-03T06:25:18.361381+0000","last_active":"2023-03-03T06:25:18.878155+0000","last_peered":"2023-03-03T06:25:18.878155+0000","last_clean":"2023-03-03T06:25:18.878155+0000","last_became_active":"2023-03-03T06:25:18.360819+0000","last_became_peered":"2023-03-03T06:25:18.360819+0000","last_unstale":"2023-03-03T06:25:18.878155+0000","last_undegraded":"2023-03-03T06:25:18.878155+0000","last_fullsized":"2023-03-03T06:25:18.878155+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:39:21.927192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335650+0000","last_change":"2023-03-03T06:25:13.848629+0000","last_active":"2023-03-03T06:25:18.335650+0000","last_peered":"2023-03-03T06:25:18.335650+0000","last_clean":"2023-03-03T06:25:18.335650+0000","last_became_active":"2023-03-03T06:25:13.848353+0000","last_became_peered":"2023-03-03T06:25:13.848353+0000","last_unstale":"2023-03-03T06:25:18.335650+0000","last_undegraded":"2023-03-03T06:25:18.335650+0000","last_fullsized":"2023-03-03T06:25:18.335650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:59:50.232210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335310+0000","last_change":"2023-03-03T06:25:13.297292+0000","last_active":"2023-03-03T06:25:18.335310+0000","last_peered":"2023-03-03T06:25:18.335310+0000","last_clean":"2023-03-03T06:25:18.335310+0000","last_became_active":"2023-03-03T06:25:13.296692+0000","last_became_peered":"2023-03-03T06:25:13.296692+0000","last_unstale":"2023-03-03T06:25:18.335310+0000","last_undegraded":"2023-03-03T06:25:18.335310+0000","last_fullsized":"2023-03-03T06:25:18.335310+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:09:43.297428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879361+0000","last_change":"2023-03-03T06:25:18.369017+0000","last_active":"2023-03-03T06:25:18.879361+0000","last_peered":"2023-03-03T06:25:18.879361+0000","last_clean":"2023-03-03T06:25:18.879361+0000","last_became_active":"2023-03-03T06:25:18.366873+0000","last_became_peered":"2023-03-03T06:25:18.366873+0000","last_unstale":"2023-03-03T06:25:18.879361+0000","last_undegraded":"2023-03-03T06:25:18.879361+0000","last_fullsized":"2023-03-03T06:25:18.879361+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:17:02.442524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334965+0000","last_change":"2023-03-03T06:25:13.855863+0000","last_active":"2023-03-03T06:25:18.334965+0000","last_peered":"2023-03-03T06:25:18.334965+0000","last_clean":"2023-03-03T06:25:18.334965+0000","last_became_active":"2023-03-03T06:25:13.855565+0000","last_became_peered":"2023-03-03T06:25:13.855565+0000","last_unstale":"2023-03-03T06:25:18.334965+0000","last_undegraded":"2023-03-03T06:25:18.334965+0000","last_fullsized":"2023-03-03T06:25:18.334965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:45:26.049658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341408+0000","last_change":"2023-03-03T06:25:13.306690+0000","last_active":"2023-03-03T06:25:18.341408+0000","last_peered":"2023-03-03T06:25:18.341408+0000","last_clean":"2023-03-03T06:25:18.341408+0000","last_became_active":"2023-03-03T06:25:13.306348+0000","last_became_peered":"2023-03-03T06:25:13.306348+0000","last_unstale":"2023-03-03T06:25:18.341408+0000","last_undegraded":"2023-03-03T06:25:18.341408+0000","last_fullsized":"2023-03-03T06:25:18.341408+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461865+0000","last_change":"2023-03-03T06:25:18.373926+0000","last_active":"2023-03-03T06:25:19.461865+0000","last_peered":"2023-03-03T06:25:19.461865+0000","last_clean":"2023-03-03T06:25:19.461865+0000","last_became_active":"2023-03-03T06:25:18.373371+0000","last_became_peered":"2023-03-03T06:25:18.373371+0000","last_unstale":"2023-03-03T06:25:19.461865+0000","last_undegraded":"2023-03-03T06:25:19.461865+0000","last_fullsized":"2023-03-03T06:25:19.461865+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:17:02.442524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335386+0000","last_change":"2023-03-03T06:25:13.846359+0000","last_active":"2023-03-03T06:25:18.335386+0000","last_peered":"2023-03-03T06:25:18.335386+0000","last_clean":"2023-03-03T06:25:18.335386+0000","last_became_active":"2023-03-03T06:25:13.845966+0000","last_became_peered":"2023-03-03T06:25:13.845966+0000","last_unstale":"2023-03-03T06:25:18.335386+0000","last_undegraded":"2023-03-03T06:25:18.335386+0000","last_fullsized":"2023-03-03T06:25:18.335386+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:38:43.138896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341996+0000","last_change":"2023-03-03T06:25:13.310274+0000","last_active":"2023-03-03T06:25:18.341996+0000","last_peered":"2023-03-03T06:25:18.341996+0000","last_clean":"2023-03-03T06:25:18.341996+0000","last_became_active":"2023-03-03T06:25:13.310005+0000","last_became_peered":"2023-03-03T06:25:13.310005+0000","last_unstale":"2023-03-03T06:25:18.341996+0000","last_undegraded":"2023-03-03T06:25:18.341996+0000","last_fullsized":"2023-03-03T06:25:18.341996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217857+0000","last_change":"2023-03-03T06:25:18.362444+0000","last_active":"2023-03-03T06:25:19.217857+0000","last_peered":"2023-03-03T06:25:19.217857+0000","last_clean":"2023-03-03T06:25:19.217857+0000","last_became_active":"2023-03-03T06:25:18.362087+0000","last_became_peered":"2023-03-03T06:25:18.362087+0000","last_unstale":"2023-03-03T06:25:19.217857+0000","last_undegraded":"2023-03-03T06:25:19.217857+0000","last_fullsized":"2023-03-03T06:25:19.217857+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:52:17.597685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343862+0000","last_change":"2023-03-03T06:25:13.834113+0000","last_active":"2023-03-03T06:25:18.343862+0000","last_peered":"2023-03-03T06:25:18.343862+0000","last_clean":"2023-03-03T06:25:18.343862+0000","last_became_active":"2023-03-03T06:25:13.833874+0000","last_became_peered":"2023-03-03T06:25:13.833874+0000","last_unstale":"2023-03-03T06:25:18.343862+0000","last_undegraded":"2023-03-03T06:25:18.343862+0000","last_fullsized":"2023-03-03T06:25:18.343862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352647+0000","last_change":"2023-03-03T06:25:13.310942+0000","last_active":"2023-03-03T06:25:18.352647+0000","last_peered":"2023-03-03T06:25:18.352647+0000","last_clean":"2023-03-03T06:25:18.352647+0000","last_became_active":"2023-03-03T06:25:13.310632+0000","last_became_peered":"2023-03-03T06:25:13.310632+0000","last_unstale":"2023-03-03T06:25:18.352647+0000","last_undegraded":"2023-03-03T06:25:18.352647+0000","last_fullsized":"2023-03-03T06:25:18.352647+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:16:16.586666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217181+0000","last_change":"2023-03-03T06:25:18.360063+0000","last_active":"2023-03-03T06:25:19.217181+0000","last_peered":"2023-03-03T06:25:19.217181+0000","last_clean":"2023-03-03T06:25:19.217181+0000","last_became_active":"2023-03-03T06:25:18.359549+0000","last_became_peered":"2023-03-03T06:25:18.359549+0000","last_unstale":"2023-03-03T06:25:19.217181+0000","last_undegraded":"2023-03-03T06:25:19.217181+0000","last_fullsized":"2023-03-03T06:25:19.217181+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:30:28.010892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341203+0000","last_change":"2023-03-03T06:25:13.844840+0000","last_active":"2023-03-03T06:25:18.341203+0000","last_peered":"2023-03-03T06:25:18.341203+0000","last_clean":"2023-03-03T06:25:18.341203+0000","last_became_active":"2023-03-03T06:25:13.844556+0000","last_became_peered":"2023-03-03T06:25:13.844556+0000","last_unstale":"2023-03-03T06:25:18.341203+0000","last_undegraded":"2023-03-03T06:25:18.341203+0000","last_fullsized":"2023-03-03T06:25:18.341203+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:19:49.547542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350702+0000","last_change":"2023-03-03T06:25:13.303131+0000","last_active":"2023-03-03T06:25:18.350702+0000","last_peered":"2023-03-03T06:25:18.350702+0000","last_clean":"2023-03-03T06:25:18.350702+0000","last_became_active":"2023-03-03T06:25:13.302861+0000","last_became_peered":"2023-03-03T06:25:13.302861+0000","last_unstale":"2023-03-03T06:25:18.350702+0000","last_undegraded":"2023-03-03T06:25:18.350702+0000","last_fullsized":"2023-03-03T06:25:18.350702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:46:07.297374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217444+0000","last_change":"2023-03-03T06:25:18.363570+0000","last_active":"2023-03-03T06:25:19.217444+0000","last_peered":"2023-03-03T06:25:19.217444+0000","last_clean":"2023-03-03T06:25:19.217444+0000","last_became_active":"2023-03-03T06:25:18.363224+0000","last_became_peered":"2023-03-03T06:25:18.363224+0000","last_unstale":"2023-03-03T06:25:19.217444+0000","last_undegraded":"2023-03-03T06:25:19.217444+0000","last_fullsized":"2023-03-03T06:25:19.217444+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:38:51.150150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335118+0000","last_change":"2023-03-03T06:25:13.837508+0000","last_active":"2023-03-03T06:25:18.335118+0000","last_peered":"2023-03-03T06:25:18.335118+0000","last_clean":"2023-03-03T06:25:18.335118+0000","last_became_active":"2023-03-03T06:25:13.836864+0000","last_became_peered":"2023-03-03T06:25:13.836864+0000","last_unstale":"2023-03-03T06:25:18.335118+0000","last_undegraded":"2023-03-03T06:25:18.335118+0000","last_fullsized":"2023-03-03T06:25:18.335118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:33:19.697671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350575+0000","last_change":"2023-03-03T06:25:13.308653+0000","last_active":"2023-03-03T06:25:18.350575+0000","last_peered":"2023-03-03T06:25:18.350575+0000","last_clean":"2023-03-03T06:25:18.350575+0000","last_became_active":"2023-03-03T06:25:13.308404+0000","last_became_peered":"2023-03-03T06:25:13.308404+0000","last_unstale":"2023-03-03T06:25:18.350575+0000","last_undegraded":"2023-03-03T06:25:18.350575+0000","last_fullsized":"2023-03-03T06:25:18.350575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:42:29.976451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217056+0000","last_change":"2023-03-03T06:25:18.369545+0000","last_active":"2023-03-03T06:25:19.217056+0000","last_peered":"2023-03-03T06:25:19.217056+0000","last_clean":"2023-03-03T06:25:19.217056+0000","last_became_active":"2023-03-03T06:25:18.369075+0000","last_became_peered":"2023-03-03T06:25:18.369075+0000","last_unstale":"2023-03-03T06:25:19.217056+0000","last_undegraded":"2023-03-03T06:25:19.217056+0000","last_fullsized":"2023-03-03T06:25:19.217056+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:45:30.096214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352873+0000","last_change":"2023-03-03T06:25:13.836258+0000","last_active":"2023-03-03T06:25:18.352873+0000","last_peered":"2023-03-03T06:25:18.352873+0000","last_clean":"2023-03-03T06:25:18.352873+0000","last_became_active":"2023-03-03T06:25:13.835993+0000","last_became_peered":"2023-03-03T06:25:13.835993+0000","last_unstale":"2023-03-03T06:25:18.352873+0000","last_undegraded":"2023-03-03T06:25:18.352873+0000","last_fullsized":"2023-03-03T06:25:18.352873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350886+0000","last_change":"2023-03-03T06:25:13.307887+0000","last_active":"2023-03-03T06:25:18.350886+0000","last_peered":"2023-03-03T06:25:18.350886+0000","last_clean":"2023-03-03T06:25:18.350886+0000","last_became_active":"2023-03-03T06:25:13.307622+0000","last_became_peered":"2023-03-03T06:25:13.307622+0000","last_unstale":"2023-03-03T06:25:18.350886+0000","last_undegraded":"2023-03-03T06:25:18.350886+0000","last_fullsized":"2023-03-03T06:25:18.350886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:51:44.342844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.462959+0000","last_change":"2023-03-03T06:25:18.366887+0000","last_active":"2023-03-03T06:25:19.462959+0000","last_peered":"2023-03-03T06:25:19.462959+0000","last_clean":"2023-03-03T06:25:19.462959+0000","last_became_active":"2023-03-03T06:25:18.366248+0000","last_became_peered":"2023-03-03T06:25:18.366248+0000","last_unstale":"2023-03-03T06:25:19.462959+0000","last_undegraded":"2023-03-03T06:25:19.462959+0000","last_fullsized":"2023-03-03T06:25:19.462959+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:06:08.404601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879274+0000","last_change":"2023-03-03T06:25:18.374976+0000","last_active":"2023-03-03T06:25:18.879274+0000","last_peered":"2023-03-03T06:25:18.879274+0000","last_clean":"2023-03-03T06:25:18.879274+0000","last_became_active":"2023-03-03T06:25:18.374414+0000","last_became_peered":"2023-03-03T06:25:18.374414+0000","last_unstale":"2023-03-03T06:25:18.879274+0000","last_undegraded":"2023-03-03T06:25:18.879274+0000","last_fullsized":"2023-03-03T06:25:18.879274+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:52:08.870718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.340487+0000","last_change":"2023-03-03T06:25:13.297905+0000","last_active":"2023-03-03T06:25:18.340487+0000","last_peered":"2023-03-03T06:25:18.340487+0000","last_clean":"2023-03-03T06:25:18.340487+0000","last_became_active":"2023-03-03T06:25:13.297568+0000","last_became_peered":"2023-03-03T06:25:13.297568+0000","last_unstale":"2023-03-03T06:25:18.340487+0000","last_undegraded":"2023-03-03T06:25:18.340487+0000","last_fullsized":"2023-03-03T06:25:18.340487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:17:34.388274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343217+0000","last_change":"2023-03-03T06:25:13.851715+0000","last_active":"2023-03-03T06:25:18.343217+0000","last_peered":"2023-03-03T06:25:18.343217+0000","last_clean":"2023-03-03T06:25:18.343217+0000","last_became_active":"2023-03-03T06:25:13.851377+0000","last_became_peered":"2023-03-03T06:25:13.851377+0000","last_unstale":"2023-03-03T06:25:18.343217+0000","last_undegraded":"2023-03-03T06:25:18.343217+0000","last_fullsized":"2023-03-03T06:25:18.343217+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467810+0000","last_change":"2023-03-03T06:25:18.356507+0000","last_active":"2023-03-03T06:25:19.467810+0000","last_peered":"2023-03-03T06:25:19.467810+0000","last_clean":"2023-03-03T06:25:19.467810+0000","last_became_active":"2023-03-03T06:25:18.356097+0000","last_became_peered":"2023-03-03T06:25:18.356097+0000","last_unstale":"2023-03-03T06:25:19.467810+0000","last_undegraded":"2023-03-03T06:25:19.467810+0000","last_fullsized":"2023-03-03T06:25:19.467810+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:59:33.700378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.351468+0000","last_change":"2023-03-03T06:25:13.306049+0000","last_active":"2023-03-03T06:25:18.351468+0000","last_peered":"2023-03-03T06:25:18.351468+0000","last_clean":"2023-03-03T06:25:18.351468+0000","last_became_active":"2023-03-03T06:25:13.305742+0000","last_became_peered":"2023-03-03T06:25:13.305742+0000","last_unstale":"2023-03-03T06:25:18.351468+0000","last_undegraded":"2023-03-03T06:25:18.351468+0000","last_fullsized":"2023-03-03T06:25:18.351468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:29:05.784203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344608+0000","last_change":"2023-03-03T06:25:13.841823+0000","last_active":"2023-03-03T06:25:18.344608+0000","last_peered":"2023-03-03T06:25:18.344608+0000","last_clean":"2023-03-03T06:25:18.344608+0000","last_became_active":"2023-03-03T06:25:13.841581+0000","last_became_peered":"2023-03-03T06:25:13.841581+0000","last_unstale":"2023-03-03T06:25:18.344608+0000","last_undegraded":"2023-03-03T06:25:18.344608+0000","last_fullsized":"2023-03-03T06:25:18.344608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342620+0000","last_change":"2023-03-03T06:25:13.307869+0000","last_active":"2023-03-03T06:25:18.342620+0000","last_peered":"2023-03-03T06:25:18.342620+0000","last_clean":"2023-03-03T06:25:18.342620+0000","last_became_active":"2023-03-03T06:25:13.303371+0000","last_became_peered":"2023-03-03T06:25:13.303371+0000","last_unstale":"2023-03-03T06:25:18.342620+0000","last_undegraded":"2023-03-03T06:25:18.342620+0000","last_fullsized":"2023-03-03T06:25:18.342620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352774+0000","last_change":"2023-03-03T06:25:13.853988+0000","last_active":"2023-03-03T06:25:18.352774+0000","last_peered":"2023-03-03T06:25:18.352774+0000","last_clean":"2023-03-03T06:25:18.352774+0000","last_became_active":"2023-03-03T06:25:13.853714+0000","last_became_peered":"2023-03-03T06:25:13.853714+0000","last_unstale":"2023-03-03T06:25:18.352774+0000","last_undegraded":"2023-03-03T06:25:18.352774+0000","last_fullsized":"2023-03-03T06:25:18.352774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:51:30.984078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467763+0000","last_change":"2023-03-03T06:25:18.372232+0000","last_active":"2023-03-03T06:25:19.467763+0000","last_peered":"2023-03-03T06:25:19.467763+0000","last_clean":"2023-03-03T06:25:19.467763+0000","last_became_active":"2023-03-03T06:25:18.371809+0000","last_became_peered":"2023-03-03T06:25:18.371809+0000","last_unstale":"2023-03-03T06:25:19.467763+0000","last_undegraded":"2023-03-03T06:25:19.467763+0000","last_fullsized":"2023-03-03T06:25:19.467763+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:11:23.940316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346535+0000","last_change":"2023-03-03T06:25:13.307069+0000","last_active":"2023-03-03T06:25:18.346535+0000","last_peered":"2023-03-03T06:25:18.346535+0000","last_clean":"2023-03-03T06:25:18.346535+0000","last_became_active":"2023-03-03T06:25:13.306791+0000","last_became_peered":"2023-03-03T06:25:13.306791+0000","last_unstale":"2023-03-03T06:25:18.346535+0000","last_undegraded":"2023-03-03T06:25:18.346535+0000","last_fullsized":"2023-03-03T06:25:18.346535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:19:48.531435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343519+0000","last_change":"2023-03-03T06:25:13.830026+0000","last_active":"2023-03-03T06:25:18.343519+0000","last_peered":"2023-03-03T06:25:18.343519+0000","last_clean":"2023-03-03T06:25:18.343519+0000","last_became_active":"2023-03-03T06:25:13.829773+0000","last_became_peered":"2023-03-03T06:25:13.829773+0000","last_unstale":"2023-03-03T06:25:18.343519+0000","last_undegraded":"2023-03-03T06:25:18.343519+0000","last_fullsized":"2023-03-03T06:25:18.343519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:35:57.127264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879167+0000","last_change":"2023-03-03T06:25:18.374930+0000","last_active":"2023-03-03T06:25:18.879167+0000","last_peered":"2023-03-03T06:25:18.879167+0000","last_clean":"2023-03-03T06:25:18.879167+0000","last_became_active":"2023-03-03T06:25:18.374374+0000","last_became_peered":"2023-03-03T06:25:18.374374+0000","last_unstale":"2023-03-03T06:25:18.879167+0000","last_undegraded":"2023-03-03T06:25:18.879167+0000","last_fullsized":"2023-03-03T06:25:18.879167+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:09:37.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.340359+0000","last_change":"2023-03-03T06:25:13.840311+0000","last_active":"2023-03-03T06:25:18.340359+0000","last_peered":"2023-03-03T06:25:18.340359+0000","last_clean":"2023-03-03T06:25:18.340359+0000","last_became_active":"2023-03-03T06:25:13.839648+0000","last_became_peered":"2023-03-03T06:25:13.839648+0000","last_unstale":"2023-03-03T06:25:18.340359+0000","last_undegraded":"2023-03-03T06:25:18.340359+0000","last_fullsized":"2023-03-03T06:25:18.340359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:29:06.800310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337721+0000","last_change":"2023-03-03T06:25:13.836213+0000","last_active":"2023-03-03T06:25:18.337721+0000","last_peered":"2023-03-03T06:25:18.337721+0000","last_clean":"2023-03-03T06:25:18.337721+0000","last_became_active":"2023-03-03T06:25:13.835890+0000","last_became_peered":"2023-03-03T06:25:13.835890+0000","last_unstale":"2023-03-03T06:25:18.337721+0000","last_undegraded":"2023-03-03T06:25:18.337721+0000","last_fullsized":"2023-03-03T06:25:18.337721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:55.025383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335018+0000","last_change":"2023-03-03T06:25:13.834786+0000","last_active":"2023-03-03T06:25:18.335018+0000","last_peered":"2023-03-03T06:25:18.335018+0000","last_clean":"2023-03-03T06:25:18.335018+0000","last_became_active":"2023-03-03T06:25:13.834531+0000","last_became_peered":"2023-03-03T06:25:13.834531+0000","last_unstale":"2023-03-03T06:25:18.335018+0000","last_undegraded":"2023-03-03T06:25:18.335018+0000","last_fullsized":"2023-03-03T06:25:18.335018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:07:17.177238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342130+0000","last_change":"2023-03-03T06:25:13.850334+0000","last_active":"2023-03-03T06:25:18.342130+0000","last_peered":"2023-03-03T06:25:18.342130+0000","last_clean":"2023-03-03T06:25:18.342130+0000","last_became_active":"2023-03-03T06:25:13.850082+0000","last_became_peered":"2023-03-03T06:25:13.850082+0000","last_unstale":"2023-03-03T06:25:18.342130+0000","last_undegraded":"2023-03-03T06:25:18.342130+0000","last_fullsized":"2023-03-03T06:25:18.342130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357706+0000","last_change":"2023-03-03T06:25:13.850690+0000","last_active":"2023-03-03T06:25:18.357706+0000","last_peered":"2023-03-03T06:25:18.357706+0000","last_clean":"2023-03-03T06:25:18.357706+0000","last_became_active":"2023-03-03T06:25:13.850392+0000","last_became_peered":"2023-03-03T06:25:13.850392+0000","last_unstale":"2023-03-03T06:25:18.357706+0000","last_undegraded":"2023-03-03T06:25:18.357706+0000","last_fullsized":"2023-03-03T06:25:18.357706+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352673+0000","last_change":"2023-03-03T06:25:13.853965+0000","last_active":"2023-03-03T06:25:18.352673+0000","last_peered":"2023-03-03T06:25:18.352673+0000","last_clean":"2023-03-03T06:25:18.352673+0000","last_became_active":"2023-03-03T06:25:13.853689+0000","last_became_peered":"2023-03-03T06:25:13.853689+0000","last_unstale":"2023-03-03T06:25:18.352673+0000","last_undegraded":"2023-03-03T06:25:18.352673+0000","last_fullsized":"2023-03-03T06:25:18.352673+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:09:17.786520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357579+0000","last_change":"2023-03-03T06:25:13.828945+0000","last_active":"2023-03-03T06:25:18.357579+0000","last_peered":"2023-03-03T06:25:18.357579+0000","last_clean":"2023-03-03T06:25:18.357579+0000","last_became_active":"2023-03-03T06:25:13.828655+0000","last_became_peered":"2023-03-03T06:25:13.828655+0000","last_unstale":"2023-03-03T06:25:18.357579+0000","last_undegraded":"2023-03-03T06:25:18.357579+0000","last_fullsized":"2023-03-03T06:25:18.357579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:51:30.984078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358304+0000","last_change":"2023-03-03T06:25:13.834307+0000","last_active":"2023-03-03T06:25:18.358304+0000","last_peered":"2023-03-03T06:25:18.358304+0000","last_clean":"2023-03-03T06:25:18.358304+0000","last_became_active":"2023-03-03T06:25:13.834054+0000","last_became_peered":"2023-03-03T06:25:13.834054+0000","last_unstale":"2023-03-03T06:25:18.358304+0000","last_undegraded":"2023-03-03T06:25:18.358304+0000","last_fullsized":"2023-03-03T06:25:18.358304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:54:02.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346796+0000","last_change":"2023-03-03T06:25:13.856830+0000","last_active":"2023-03-03T06:25:18.346796+0000","last_peered":"2023-03-03T06:25:18.346796+0000","last_clean":"2023-03-03T06:25:18.346796+0000","last_became_active":"2023-03-03T06:25:13.856494+0000","last_became_peered":"2023-03-03T06:25:13.856494+0000","last_unstale":"2023-03-03T06:25:18.346796+0000","last_undegraded":"2023-03-03T06:25:18.346796+0000","last_fullsized":"2023-03-03T06:25:18.346796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:59:50.232210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337609+0000","last_change":"2023-03-03T06:25:13.836186+0000","last_active":"2023-03-03T06:25:18.337609+0000","last_peered":"2023-03-03T06:25:18.337609+0000","last_clean":"2023-03-03T06:25:18.337609+0000","last_became_active":"2023-03-03T06:25:13.835481+0000","last_became_peered":"2023-03-03T06:25:13.835481+0000","last_unstale":"2023-03-03T06:25:18.337609+0000","last_undegraded":"2023-03-03T06:25:18.337609+0000","last_fullsized":"2023-03-03T06:25:18.337609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:30:23.964335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342530+0000","last_change":"2023-03-03T06:25:13.834687+0000","last_active":"2023-03-03T06:25:18.342530+0000","last_peered":"2023-03-03T06:25:18.342530+0000","last_clean":"2023-03-03T06:25:18.342530+0000","last_became_active":"2023-03-03T06:25:13.834424+0000","last_became_peered":"2023-03-03T06:25:13.834424+0000","last_unstale":"2023-03-03T06:25:18.342530+0000","last_undegraded":"2023-03-03T06:25:18.342530+0000","last_fullsized":"2023-03-03T06:25:18.342530+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346438+0000","last_change":"2023-03-03T06:25:13.855849+0000","last_active":"2023-03-03T06:25:18.346438+0000","last_peered":"2023-03-03T06:25:18.346438+0000","last_clean":"2023-03-03T06:25:18.346438+0000","last_became_active":"2023-03-03T06:25:13.855609+0000","last_became_peered":"2023-03-03T06:25:13.855609+0000","last_unstale":"2023-03-03T06:25:18.346438+0000","last_undegraded":"2023-03-03T06:25:18.346438+0000","last_fullsized":"2023-03-03T06:25:18.346438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335348+0000","last_change":"2023-03-03T06:25:13.846670+0000","last_active":"2023-03-03T06:25:18.335348+0000","last_peered":"2023-03-03T06:25:18.335348+0000","last_clean":"2023-03-03T06:25:18.335348+0000","last_became_active":"2023-03-03T06:25:13.846349+0000","last_became_peered":"2023-03-03T06:25:13.846349+0000","last_unstale":"2023-03-03T06:25:18.335348+0000","last_undegraded":"2023-03-03T06:25:18.335348+0000","last_fullsized":"2023-03-03T06:25:18.335348+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:30:23.964335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337310+0000","last_change":"2023-03-03T06:25:13.839483+0000","last_active":"2023-03-03T06:25:18.337310+0000","last_peered":"2023-03-03T06:25:18.337310+0000","last_clean":"2023-03-03T06:25:18.337310+0000","last_became_active":"2023-03-03T06:25:13.838497+0000","last_became_peered":"2023-03-03T06:25:13.838497+0000","last_unstale":"2023-03-03T06:25:18.337310+0000","last_undegraded":"2023-03-03T06:25:18.337310+0000","last_fullsized":"2023-03-03T06:25:18.337310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:08.122967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334979+0000","last_change":"2023-03-03T06:25:13.855262+0000","last_active":"2023-03-03T06:25:18.334979+0000","last_peered":"2023-03-03T06:25:18.334979+0000","last_clean":"2023-03-03T06:25:18.334979+0000","last_became_active":"2023-03-03T06:25:13.848660+0000","last_became_peered":"2023-03-03T06:25:13.848660+0000","last_unstale":"2023-03-03T06:25:18.334979+0000","last_undegraded":"2023-03-03T06:25:18.334979+0000","last_fullsized":"2023-03-03T06:25:18.334979+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:55.025383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344570+0000","last_change":"2023-03-03T06:25:13.842845+0000","last_active":"2023-03-03T06:25:18.344570+0000","last_peered":"2023-03-03T06:25:18.344570+0000","last_clean":"2023-03-03T06:25:18.344570+0000","last_became_active":"2023-03-03T06:25:13.842580+0000","last_became_peered":"2023-03-03T06:25:13.842580+0000","last_unstale":"2023-03-03T06:25:18.344570+0000","last_undegraded":"2023-03-03T06:25:18.344570+0000","last_fullsized":"2023-03-03T06:25:18.344570+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:28:42.514217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335348+0000","last_change":"2023-03-03T06:25:13.836306+0000","last_active":"2023-03-03T06:25:18.335348+0000","last_peered":"2023-03-03T06:25:18.335348+0000","last_clean":"2023-03-03T06:25:18.335348+0000","last_became_active":"2023-03-03T06:25:13.835953+0000","last_became_peered":"2023-03-03T06:25:13.835953+0000","last_unstale":"2023-03-03T06:25:18.335348+0000","last_undegraded":"2023-03-03T06:25:18.335348+0000","last_fullsized":"2023-03-03T06:25:18.335348+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:35:57.127264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.351349+0000","last_change":"2023-03-03T06:25:13.854132+0000","last_active":"2023-03-03T06:25:18.351349+0000","last_peered":"2023-03-03T06:25:18.351349+0000","last_clean":"2023-03-03T06:25:18.351349+0000","last_became_active":"2023-03-03T06:25:13.853786+0000","last_became_peered":"2023-03-03T06:25:13.853786+0000","last_unstale":"2023-03-03T06:25:18.351349+0000","last_undegraded":"2023-03-03T06:25:18.351349+0000","last_fullsized":"2023-03-03T06:25:18.351349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353001+0000","last_change":"2023-03-03T06:25:13.835134+0000","last_active":"2023-03-03T06:25:18.353001+0000","last_peered":"2023-03-03T06:25:18.353001+0000","last_clean":"2023-03-03T06:25:18.353001+0000","last_became_active":"2023-03-03T06:25:13.834844+0000","last_became_peered":"2023-03-03T06:25:13.834844+0000","last_unstale":"2023-03-03T06:25:18.353001+0000","last_undegraded":"2023-03-03T06:25:18.353001+0000","last_fullsized":"2023-03-03T06:25:18.353001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337727+0000","last_change":"2023-03-03T06:25:13.830031+0000","last_active":"2023-03-03T06:25:18.337727+0000","last_peered":"2023-03-03T06:25:18.337727+0000","last_clean":"2023-03-03T06:25:18.337727+0000","last_became_active":"2023-03-03T06:25:13.829770+0000","last_became_peered":"2023-03-03T06:25:13.829770+0000","last_unstale":"2023-03-03T06:25:18.337727+0000","last_undegraded":"2023-03-03T06:25:18.337727+0000","last_fullsized":"2023-03-03T06:25:18.337727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:06.900541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.336311+0000","last_change":"2023-03-03T06:25:13.858248+0000","last_active":"2023-03-03T06:25:18.336311+0000","last_peered":"2023-03-03T06:25:18.336311+0000","last_clean":"2023-03-03T06:25:18.336311+0000","last_became_active":"2023-03-03T06:25:13.857884+0000","last_became_peered":"2023-03-03T06:25:13.857884+0000","last_unstale":"2023-03-03T06:25:18.336311+0000","last_undegraded":"2023-03-03T06:25:18.336311+0000","last_fullsized":"2023-03-03T06:25:18.336311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:39.051115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334811+0000","last_change":"2023-03-03T06:25:13.847616+0000","last_active":"2023-03-03T06:25:18.334811+0000","last_peered":"2023-03-03T06:25:18.334811+0000","last_clean":"2023-03-03T06:25:18.334811+0000","last_became_active":"2023-03-03T06:25:13.847110+0000","last_became_peered":"2023-03-03T06:25:13.847110+0000","last_unstale":"2023-03-03T06:25:18.334811+0000","last_undegraded":"2023-03-03T06:25:18.334811+0000","last_fullsized":"2023-03-03T06:25:18.334811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:52:13.551129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353150+0000","last_change":"2023-03-03T06:25:13.851731+0000","last_active":"2023-03-03T06:25:18.353150+0000","last_peered":"2023-03-03T06:25:18.353150+0000","last_clean":"2023-03-03T06:25:18.353150+0000","last_became_active":"2023-03-03T06:25:13.849868+0000","last_became_peered":"2023-03-03T06:25:13.849868+0000","last_unstale":"2023-03-03T06:25:18.353150+0000","last_undegraded":"2023-03-03T06:25:18.353150+0000","last_fullsized":"2023-03-03T06:25:18.353150+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343321+0000","last_change":"2023-03-03T06:25:13.846247+0000","last_active":"2023-03-03T06:25:18.343321+0000","last_peered":"2023-03-03T06:25:18.343321+0000","last_clean":"2023-03-03T06:25:18.343321+0000","last_became_active":"2023-03-03T06:25:13.845983+0000","last_became_peered":"2023-03-03T06:25:13.845983+0000","last_unstale":"2023-03-03T06:25:18.343321+0000","last_undegraded":"2023-03-03T06:25:18.343321+0000","last_fullsized":"2023-03-03T06:25:18.343321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:54:02.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352766+0000","last_change":"2023-03-03T06:25:13.858013+0000","last_active":"2023-03-03T06:25:18.352766+0000","last_peered":"2023-03-03T06:25:18.352766+0000","last_clean":"2023-03-03T06:25:18.352766+0000","last_became_active":"2023-03-03T06:25:13.857706+0000","last_became_peered":"2023-03-03T06:25:13.857706+0000","last_unstale":"2023-03-03T06:25:18.352766+0000","last_undegraded":"2023-03-03T06:25:18.352766+0000","last_fullsized":"2023-03-03T06:25:18.352766+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:54:02.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.351182+0000","last_change":"2023-03-03T06:25:13.854691+0000","last_active":"2023-03-03T06:25:18.351182+0000","last_peered":"2023-03-03T06:25:18.351182+0000","last_clean":"2023-03-03T06:25:18.351182+0000","last_became_active":"2023-03-03T06:25:13.854332+0000","last_became_peered":"2023-03-03T06:25:13.854332+0000","last_unstale":"2023-03-03T06:25:18.351182+0000","last_undegraded":"2023-03-03T06:25:18.351182+0000","last_fullsized":"2023-03-03T06:25:18.351182+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334771+0000","last_change":"2023-03-03T06:25:13.836791+0000","last_active":"2023-03-03T06:25:18.334771+0000","last_peered":"2023-03-03T06:25:18.334771+0000","last_clean":"2023-03-03T06:25:18.334771+0000","last_became_active":"2023-03-03T06:25:13.836500+0000","last_became_peered":"2023-03-03T06:25:13.836500+0000","last_unstale":"2023-03-03T06:25:18.334771+0000","last_undegraded":"2023-03-03T06:25:18.334771+0000","last_fullsized":"2023-03-03T06:25:18.334771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:38:47.103593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344030+0000","last_change":"2023-03-03T06:25:13.846407+0000","last_active":"2023-03-03T06:25:18.344030+0000","last_peered":"2023-03-03T06:25:18.344030+0000","last_clean":"2023-03-03T06:25:18.344030+0000","last_became_active":"2023-03-03T06:25:13.846119+0000","last_became_peered":"2023-03-03T06:25:13.846119+0000","last_unstale":"2023-03-03T06:25:18.344030+0000","last_undegraded":"2023-03-03T06:25:18.344030+0000","last_fullsized":"2023-03-03T06:25:18.344030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341377+0000","last_change":"2023-03-03T06:25:13.843673+0000","last_active":"2023-03-03T06:25:18.341377+0000","last_peered":"2023-03-03T06:25:18.341377+0000","last_clean":"2023-03-03T06:25:18.341377+0000","last_became_active":"2023-03-03T06:25:13.843408+0000","last_became_peered":"2023-03-03T06:25:13.843408+0000","last_unstale":"2023-03-03T06:25:18.341377+0000","last_undegraded":"2023-03-03T06:25:18.341377+0000","last_fullsized":"2023-03-03T06:25:18.341377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:51:45.358951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335571+0000","last_change":"2023-03-03T06:25:13.847791+0000","last_active":"2023-03-03T06:25:18.335571+0000","last_peered":"2023-03-03T06:25:18.335571+0000","last_clean":"2023-03-03T06:25:18.335571+0000","last_became_active":"2023-03-03T06:25:13.847378+0000","last_became_peered":"2023-03-03T06:25:13.847378+0000","last_unstale":"2023-03-03T06:25:18.335571+0000","last_undegraded":"2023-03-03T06:25:18.335571+0000","last_fullsized":"2023-03-03T06:25:18.335571+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:05:53.754239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353098+0000","last_change":"2023-03-03T06:25:13.835727+0000","last_active":"2023-03-03T06:25:18.353098+0000","last_peered":"2023-03-03T06:25:18.353098+0000","last_clean":"2023-03-03T06:25:18.353098+0000","last_became_active":"2023-03-03T06:25:13.835462+0000","last_became_peered":"2023-03-03T06:25:13.835462+0000","last_unstale":"2023-03-03T06:25:18.353098+0000","last_undegraded":"2023-03-03T06:25:18.353098+0000","last_fullsized":"2023-03-03T06:25:18.353098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.336596+0000","last_change":"2023-03-03T06:25:13.827811+0000","last_active":"2023-03-03T06:25:18.336596+0000","last_peered":"2023-03-03T06:25:18.336596+0000","last_clean":"2023-03-03T06:25:18.336596+0000","last_became_active":"2023-03-03T06:25:13.827532+0000","last_became_peered":"2023-03-03T06:25:13.827532+0000","last_unstale":"2023-03-03T06:25:18.336596+0000","last_undegraded":"2023-03-03T06:25:18.336596+0000","last_fullsized":"2023-03-03T06:25:18.336596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:09.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4527,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13456,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":12544,"kb_avail":94358384,"statfs":{"total":96636764160,"available":96622985216,"internally_reserved":0,"allocated":925696,"data_stored":718901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12845056},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.639}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12272,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":11968,"kb_avail":94359568,"statfs":{"total":96636764160,"available":96624197632,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12255232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.772}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6519999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.577}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12280,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":11968,"kb_avail":94359560,"statfs":{"total":96636764160,"available":96624189440,"internally_reserved":0,"allocated":311296,"data_stored":125208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12255232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3540000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.609}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7989999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12356,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":12032,"kb_avail":94359484,"statfs":{"total":96636764160,"available":96624111616,"internally_reserved":0,"allocated":323584,"data_stored":128373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12320768},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.607}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6359999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8112,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363728,"statfs":{"total":96636764160,"available":96628457472,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7010000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8240,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363600,"statfs":{"total":96636764160,"available":96628326400,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.726}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.591}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.546}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9272,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362568,"statfs":{"total":96636764160,"available":96627269632,"internally_reserved":0,"allocated":901120,"data_stored":715506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.956}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8380000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-03T06:25:43.774 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-03-03T06:25:43.775 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-03-03T06:25:43.775 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-03-03T06:25:43.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-03-03T06:25:43.776 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-03-03T06:25:44.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.019+0000 7fa82d949700 1 Processor -- start 2023-03-03T06:25:44.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.023+0000 7fa82d949700 1 -- start start 2023-03-03T06:25:44.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.023+0000 7fa82d949700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8281585d0 0x7fa82806fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.023+0000 7fa82d949700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8280703b0 0x7fa8280cca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.023+0000 7fa82d949700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8280ccfd0 0x7fa8280b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.023+0000 7fa82d949700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8280cd450 con 0x7fa8280ccfd0 2023-03-03T06:25:44.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.023+0000 7fa82d949700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82815c180 con 0x7fa8281585d0 2023-03-03T06:25:44.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.023+0000 7fa82d949700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82815c460 con 0x7fa8280703b0 2023-03-03T06:25:44.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa827fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8280ccfd0 0x7fa8280b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa827fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8280ccfd0 0x7fa8280b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43304/0 (socket says 172.21.15.3:43304) 2023-03-03T06:25:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa827fff700 1 -- 172.21.15.3:0/4165245335 learned_addr learned my addr 172.21.15.3:0/4165245335 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa826ffd700 1 --2- 172.21.15.3:0/4165245335 >> v2:172.21.15.32:3300/0 conn(0x7fa8281585d0 0x7fa82806fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa8267fc700 1 --2- 172.21.15.3:0/4165245335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8280703b0 0x7fa8280cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa827fff700 1 -- 172.21.15.3:0/4165245335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8280703b0 msgr2=0x7fa8280cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa827fff700 1 --2- 172.21.15.3:0/4165245335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8280703b0 0x7fa8280cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa827fff700 1 -- 172.21.15.3:0/4165245335 >> v2:172.21.15.32:3300/0 conn(0x7fa8281585d0 msgr2=0x7fa82806fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa827fff700 1 --2- 172.21.15.3:0/4165245335 >> v2:172.21.15.32:3300/0 conn(0x7fa8281585d0 0x7fa82806fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa827fff700 1 -- 172.21.15.3:0/4165245335 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82815c740 con 0x7fa8280ccfd0 2023-03-03T06:25:44.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa827fff700 1 --2- 172.21.15.3:0/4165245335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8280ccfd0 0x7fa8280b1c60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa8280b2da0 tx=0x7fa82000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12fded096ded2e4 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:44.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa825ffb700 1 -- 172.21.15.3:0/4165245335 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82000e070 con 0x7fa8280ccfd0 2023-03-03T06:25:44.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa825ffb700 1 -- 172.21.15.3:0/4165245335 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82000b9c0 con 0x7fa8280ccfd0 2023-03-03T06:25:44.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa825ffb700 1 -- 172.21.15.3:0/4165245335 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82000bd70 con 0x7fa8280ccfd0 2023-03-03T06:25:44.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa82d949700 1 -- 172.21.15.3:0/4165245335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8280ccfd0 msgr2=0x7fa8280b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa82d949700 1 --2- 172.21.15.3:0/4165245335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8280ccfd0 0x7fa8280b1c60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa8280b2da0 tx=0x7fa82000b330 comp rx=0 tx=0).stop 2023-03-03T06:25:44.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa82d949700 1 -- 172.21.15.3:0/4165245335 shutdown_connections 2023-03-03T06:25:44.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa82d949700 1 --2- 172.21.15.3:0/4165245335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8280ccfd0 0x7fa8280b1c60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa82d949700 1 --2- 172.21.15.3:0/4165245335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8280703b0 0x7fa8280cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa82d949700 1 --2- 172.21.15.3:0/4165245335 >> v2:172.21.15.32:3300/0 conn(0x7fa8281585d0 0x7fa82806fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa82d949700 1 -- 172.21.15.3:0/4165245335 >> 172.21.15.3:0/4165245335 conn(0x7fa82806b580 msgr2=0x7fa82806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa82d949700 1 -- 172.21.15.3:0/4165245335 shutdown_connections 2023-03-03T06:25:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.027+0000 7fa82d949700 1 -- 172.21.15.3:0/4165245335 wait complete. 2023-03-03T06:25:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.031+0000 7fa82d949700 1 Processor -- start 2023-03-03T06:25:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.031+0000 7fa82d949700 1 -- start start 2023-03-03T06:25:44.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.031+0000 7fa82d949700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8280703b0 0x7fa8280cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.031+0000 7fa82d949700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8280ccfd0 0x7fa8281589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.031+0000 7fa82d949700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8281585d0 0x7fa828158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.031+0000 7fa82d949700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8280ced00 con 0x7fa8281585d0 2023-03-03T06:25:44.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.031+0000 7fa82d949700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8280cefe0 con 0x7fa8280703b0 2023-03-03T06:25:44.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.031+0000 7fa82d949700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8280cf2c0 con 0x7fa8280ccfd0 2023-03-03T06:25:44.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa827fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8281585d0 0x7fa828158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa827fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8281585d0 0x7fa828158ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43312/0 (socket says 172.21.15.3:43312) 2023-03-03T06:25:44.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa827fff700 1 -- 172.21.15.3:0/230249218 learned_addr learned my addr 172.21.15.3:0/230249218 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:44.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa8267fc700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8280ccfd0 0x7fa8281589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa826ffd700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8280703b0 0x7fa8280cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa827fff700 1 -- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8280ccfd0 msgr2=0x7fa8281589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa827fff700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8280ccfd0 0x7fa8281589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa827fff700 1 -- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8280703b0 msgr2=0x7fa8280cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa827fff700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8280703b0 0x7fa8280cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa827fff700 1 -- 172.21.15.3:0/230249218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa820009b30 con 0x7fa8281585d0 2023-03-03T06:25:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa827fff700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8281585d0 0x7fa828158ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa828071120 tx=0x7fa820004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:44.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa80ffff700 1 -- 172.21.15.3:0/230249218 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820004250 con 0x7fa8281585d0 2023-03-03T06:25:44.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa80ffff700 1 -- 172.21.15.3:0/230249218 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8200043b0 con 0x7fa8281585d0 2023-03-03T06:25:44.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa80ffff700 1 -- 172.21.15.3:0/230249218 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820004760 con 0x7fa8281585d0 2023-03-03T06:25:44.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa82d949700 1 -- 172.21.15.3:0/230249218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82815cd00 con 0x7fa8281585d0 2023-03-03T06:25:44.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.035+0000 7fa82d949700 1 -- 172.21.15.3:0/230249218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82815ca20 con 0x7fa8281585d0 2023-03-03T06:25:44.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.039+0000 7fa82d949700 1 -- 172.21.15.3:0/230249218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82815cfe0 con 0x7fa8281585d0 2023-03-03T06:25:44.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.039+0000 7fa80ffff700 1 -- 172.21.15.3:0/230249218 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa82002c070 con 0x7fa8281585d0 2023-03-03T06:25:44.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.043+0000 7fa80ffff700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa814064ee0 0x7fa814067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.043+0000 7fa80ffff700 1 -- 172.21.15.3:0/230249218 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa820083aa0 con 0x7fa8281585d0 2023-03-03T06:25:44.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.043+0000 7fa826ffd700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa814064ee0 0x7fa814067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.043+0000 7fa826ffd700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa814064ee0 0x7fa814067390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa81800b9d0 tx=0x7fa818008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:44.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.047+0000 7fa80ffff700 1 -- 172.21.15.3:0/230249218 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa8200517e0 con 0x7fa8281585d0 2023-03-03T06:25:44.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.263+0000 7fa82d949700 1 -- 172.21.15.3:0/230249218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa82804e180 con 0x7fa8281585d0 2023-03-03T06:25:44.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.271+0000 7fa80ffff700 1 -- 172.21.15.3:0/230249218 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+161629 (secure 0 0 0) 0x7fa820054e00 con 0x7fa8281585d0 2023-03-03T06:25:44.278 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:44.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.275+0000 7fa82d949700 1 -- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa814064ee0 msgr2=0x7fa814067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.275+0000 7fa82d949700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa814064ee0 0x7fa814067390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa81800b9d0 tx=0x7fa818008040 comp rx=0 tx=0).stop 2023-03-03T06:25:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.275+0000 7fa82d949700 1 -- 172.21.15.3:0/230249218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8281585d0 msgr2=0x7fa828158ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.275+0000 7fa82d949700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8281585d0 0x7fa828158ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa828071120 tx=0x7fa820004af0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.275+0000 7fa82d949700 1 -- 172.21.15.3:0/230249218 shutdown_connections 2023-03-03T06:25:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.275+0000 7fa82d949700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa814064ee0 0x7fa814067390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.275+0000 7fa82d949700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8281585d0 0x7fa828158ee0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.275+0000 7fa82d949700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8280ccfd0 0x7fa8281589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.275+0000 7fa82d949700 1 --2- 172.21.15.3:0/230249218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8280703b0 0x7fa8280cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.275+0000 7fa82d949700 1 -- 172.21.15.3:0/230249218 >> 172.21.15.3:0/230249218 conn(0x7fa82806b580 msgr2=0x7fa8280b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:44.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.279+0000 7fa82d949700 1 -- 172.21.15.3:0/230249218 shutdown_connections 2023-03-03T06:25:44.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.279+0000 7fa82d949700 1 -- 172.21.15.3:0/230249218 wait complete. 2023-03-03T06:25:44.314 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":32113},{"type":"v1","addr":"172.21.15.32:6833","nonce":32113}]},"active_addr":"172.21.15.32:6833/32113","active_change":"2023-03-03T06:24:52.363161+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3308378041}]},{"addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":672402080}]},{"addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1428587638}]},{"addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2546625418}]}]} 2023-03-03T06:25:44.320 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-03-03T06:25:44.320 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-03-03T06:25:44.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:25:44.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.523+0000 7f4aa4e98700 1 Processor -- start 2023-03-03T06:25:44.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.523+0000 7f4aa4e98700 1 -- start start 2023-03-03T06:25:44.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4aa4e98700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4aa01608d0 0x7f4aa00cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4aa4e98700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4aa00cfb60 0x7f4aa00d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4aa4e98700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa0160440 con 0x7f4aa0153670 2023-03-03T06:25:44.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4aa4e98700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa00cffe0 con 0x7f4aa01608d0 2023-03-03T06:25:44.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4aa4e98700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa00cd3e0 con 0x7f4aa00cfb60 2023-03-03T06:25:44.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4a96ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4aa01608d0 0x7f4aa00cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4a96ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4aa01608d0 0x7f4aa00cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43318/0 (socket says 172.21.15.3:43318) 2023-03-03T06:25:44.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4a96ffd700 1 -- 172.21.15.3:0/1173316045 learned_addr learned my addr 172.21.15.3:0/1173316045 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:44.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4a9ffff700 1 --2- 172.21.15.3:0/1173316045 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4aa00cfb60 0x7f4aa00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4a96ffd700 1 -- 172.21.15.3:0/1173316045 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4aa00cfb60 msgr2=0x7f4aa00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:44.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4a96ffd700 1 --2- 172.21.15.3:0/1173316045 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4aa00cfb60 0x7f4aa00d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4a96ffd700 1 -- 172.21.15.3:0/1173316045 >> v1:172.21.15.3:6789/0 conn(0x7f4aa0153670 legacy=0x7f4aa0071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4a96ffd700 1 -- 172.21.15.3:0/1173316045 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa00cd6c0 con 0x7f4aa01608d0 2023-03-03T06:25:44.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4a9ffff700 1 --2- 172.21.15.3:0/1173316045 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4aa00cfb60 0x7f4aa00d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:25:44.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.527+0000 7f4a96ffd700 1 --2- 172.21.15.3:0/1173316045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa01608d0 0x7f4aa00cf620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009940 tx=0x7f4a88009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b214da08a59c23d0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:44.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.531+0000 7f4a9effd700 1 -- 172.21.15.3:0/1173316045 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8800ba20 con 0x7f4aa01608d0 2023-03-03T06:25:44.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.531+0000 7f4a9effd700 1 -- 172.21.15.3:0/1173316045 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a88009e90 con 0x7f4aa01608d0 2023-03-03T06:25:44.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.531+0000 7f4a9effd700 1 -- 172.21.15.3:0/1173316045 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8800bda0 con 0x7f4aa01608d0 2023-03-03T06:25:44.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.531+0000 7f4aa4e98700 1 -- 172.21.15.3:0/1173316045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa01608d0 msgr2=0x7f4aa00cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.531+0000 7f4aa4e98700 1 --2- 172.21.15.3:0/1173316045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa01608d0 0x7f4aa00cf620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009940 tx=0x7f4a88009c50 comp rx=0 tx=0).stop 2023-03-03T06:25:44.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.531+0000 7f4aa4e98700 1 -- 172.21.15.3:0/1173316045 shutdown_connections 2023-03-03T06:25:44.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.531+0000 7f4aa4e98700 1 --2- 172.21.15.3:0/1173316045 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4aa00cfb60 0x7f4aa00d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.531+0000 7f4aa4e98700 1 --2- 172.21.15.3:0/1173316045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa01608d0 0x7f4aa00cf620 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.531+0000 7f4aa4e98700 1 -- 172.21.15.3:0/1173316045 >> 172.21.15.3:0/1173316045 conn(0x7f4aa006e5c0 msgr2=0x7f4aa006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:44.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.531+0000 7f4aa4e98700 1 -- 172.21.15.3:0/1173316045 shutdown_connections 2023-03-03T06:25:44.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.531+0000 7f4aa4e98700 1 -- 172.21.15.3:0/1173316045 wait complete. 2023-03-03T06:25:44.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4aa4e98700 1 Processor -- start 2023-03-03T06:25:44.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4aa4e98700 1 -- start start 2023-03-03T06:25:44.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4aa4e98700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4aa00cfb60 0x7f4aa00b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4aa4e98700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa0153670 0x7f4aa00cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4aa4e98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa01608d0 0x7f4aa00b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4aa4e98700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa01f1ca0 con 0x7f4aa0153670 2023-03-03T06:25:44.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4aa4e98700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa01f5790 con 0x7f4aa01608d0 2023-03-03T06:25:44.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4aa4e98700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa01f5a70 con 0x7f4aa00cfb60 2023-03-03T06:25:44.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4a96ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa0153670 0x7f4aa00cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4a96ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa0153670 0x7f4aa00cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43324/0 (socket says 172.21.15.3:43324) 2023-03-03T06:25:44.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4a96ffd700 1 -- 172.21.15.3:0/3522630378 learned_addr learned my addr 172.21.15.3:0/3522630378 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:44.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4a9f7fe700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4aa00cfb60 0x7f4aa00b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4a9ffff700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa01608d0 0x7f4aa00b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.535+0000 7f4a96ffd700 1 -- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4aa00cfb60 msgr2=0x7f4aa00b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.539+0000 7f4a96ffd700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4aa00cfb60 0x7f4aa00b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.539+0000 7f4a96ffd700 1 -- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa01608d0 msgr2=0x7f4aa00b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.539+0000 7f4a96ffd700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa01608d0 0x7f4aa00b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.539+0000 7f4a96ffd700 1 -- 172.21.15.3:0/3522630378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a88009570 con 0x7f4aa0153670 2023-03-03T06:25:44.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.539+0000 7f4a9ffff700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa01608d0 0x7f4aa00b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:25:44.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.539+0000 7f4a96ffd700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa0153670 0x7f4aa00cc530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009910 tx=0x7f4a88004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:44.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.539+0000 7f4a9cff9700 1 -- 172.21.15.3:0/3522630378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88004460 con 0x7f4aa0153670 2023-03-03T06:25:44.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.539+0000 7f4a9cff9700 1 -- 172.21.15.3:0/3522630378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8801e070 con 0x7f4aa0153670 2023-03-03T06:25:44.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.539+0000 7f4aa4e98700 1 -- 172.21.15.3:0/3522630378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa00cd9a0 con 0x7f4aa0153670 2023-03-03T06:25:44.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.539+0000 7f4a9cff9700 1 -- 172.21.15.3:0/3522630378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88012950 con 0x7f4aa0153670 2023-03-03T06:25:44.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.539+0000 7f4aa4e98700 1 -- 172.21.15.3:0/3522630378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa01600d0 con 0x7f4aa0153670 2023-03-03T06:25:44.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.539+0000 7f4aa4e98700 1 -- 172.21.15.3:0/3522630378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa00cdb30 con 0x7f4aa0153670 2023-03-03T06:25:44.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.543+0000 7f4a9cff9700 1 -- 172.21.15.3:0/3522630378 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4a8800f050 con 0x7f4aa0153670 2023-03-03T06:25:44.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.547+0000 7f4a9cff9700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4a80064f30 0x7f4a800673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.547+0000 7f4a9cff9700 1 -- 172.21.15.3:0/3522630378 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4a88020070 con 0x7f4aa0153670 2023-03-03T06:25:44.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.547+0000 7f4a9f7fe700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4a80064f30 0x7f4a800673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.547+0000 7f4a9f7fe700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4a80064f30 0x7f4a800673e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4a90000f60 tx=0x7f4a90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:44.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.555+0000 7f4a9cff9700 1 -- 172.21.15.3:0/3522630378 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4a880517b0 con 0x7f4aa0153670 2023-03-03T06:25:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.727+0000 7f4aa4e98700 1 -- 172.21.15.3:0/3522630378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4aa01f1de0 con 0x7f4aa0153670 2023-03-03T06:25:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.731+0000 7f4a9cff9700 1 -- 172.21.15.3:0/3522630378 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19556 (secure 0 0 0) 0x7f4a88054dd0 con 0x7f4aa0153670 2023-03-03T06:25:44.735 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:44.736 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:18.863350+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:25:44.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.735+0000 7f4aa4e98700 1 -- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4a80064f30 msgr2=0x7f4a800673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.735+0000 7f4aa4e98700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4a80064f30 0x7f4a800673e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4a90000f60 tx=0x7f4a90009040 comp rx=0 tx=0).stop 2023-03-03T06:25:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.735+0000 7f4aa4e98700 1 -- 172.21.15.3:0/3522630378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa0153670 msgr2=0x7f4aa00cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.735+0000 7f4aa4e98700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa0153670 0x7f4aa00cc530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009910 tx=0x7f4a88004770 comp rx=0 tx=0).stop 2023-03-03T06:25:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.735+0000 7f4aa4e98700 1 -- 172.21.15.3:0/3522630378 shutdown_connections 2023-03-03T06:25:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.735+0000 7f4aa4e98700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4a80064f30 0x7f4a800673e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.735+0000 7f4aa4e98700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa01608d0 0x7f4aa00b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.735+0000 7f4aa4e98700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa0153670 0x7f4aa00cc530 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.735+0000 7f4aa4e98700 1 --2- 172.21.15.3:0/3522630378 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4aa00cfb60 0x7f4aa00b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.735+0000 7f4aa4e98700 1 -- 172.21.15.3:0/3522630378 >> 172.21.15.3:0/3522630378 conn(0x7f4aa006e5c0 msgr2=0x7f4aa015db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.735+0000 7f4aa4e98700 1 -- 172.21.15.3:0/3522630378 shutdown_connections 2023-03-03T06:25:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.735+0000 7f4aa4e98700 1 -- 172.21.15.3:0/3522630378 wait complete. 2023-03-03T06:25:44.775 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-03-03T06:25:44.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:25:44.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.975+0000 7f4d35bdf700 1 Processor -- start 2023-03-03T06:25:44.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.975+0000 7f4d35bdf700 1 -- start start 2023-03-03T06:25:44.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.979+0000 7f4d35bdf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d300adcd0 0x7f4d300cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.979+0000 7f4d35bdf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d300cbb40 con 0x7f4d30158ab0 2023-03-03T06:25:44.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.979+0000 7f4d35bdf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d300cceb0 con 0x7f4d300cb6c0 2023-03-03T06:25:44.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.979+0000 7f4d35bdf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d300ce3f0 con 0x7f4d300adcd0 2023-03-03T06:25:44.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.979+0000 7f4d34bdd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4d30158ab0 0x7f4d300cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33660/0 (socket says 172.21.15.3:33660) 2023-03-03T06:25:44.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.979+0000 7f4d34bdd700 1 -- 172.21.15.3:0/2567160959 learned_addr learned my addr 172.21.15.3:0/2567160959 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:44.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.979+0000 7f4d27fff700 1 --2- 172.21.15.3:0/2567160959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d300adcd0 0x7f4d300cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.979+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963255883 0 0) 0x7f4d2c001110 con 0x7f4d30158ab0 2023-03-03T06:25:44.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.979+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d180035d0 con 0x7f4d30158ab0 2023-03-03T06:25:44.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2018909134 0 0) 0x7f4d180035d0 con 0x7f4d30158ab0 2023-03-03T06:25:44.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d300adcd0 msgr2=0x7f4d300cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:44.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 --2- 172.21.15.3:0/2567160959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d300adcd0 0x7f4d300cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 >> v1:172.21.15.32:6789/0 conn(0x7f4d300cb6c0 legacy=0x7f4d300af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d300cd340 con 0x7f4d30158ab0 2023-03-03T06:25:44.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f4d2c001800 con 0x7f4d30158ab0 2023-03-03T06:25:44.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 >> v1:172.21.15.3:6789/0 conn(0x7f4d30158ab0 legacy=0x7f4d300cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 --2- 172.21.15.3:0/2567160959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d18003a50 0x7f4d18003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 --2- 172.21.15.3:0/2567160959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d18004870 0x7f4d18004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 --2- 172.21.15.3:0/2567160959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d18005220 0x7f4d1800cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1800d1c0 con 0x7f4d18004870 2023-03-03T06:25:44.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1800d300 con 0x7f4d18005220 2023-03-03T06:25:44.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1800d4c0 con 0x7f4d18003a50 2023-03-03T06:25:44.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d27fff700 1 --2- 172.21.15.3:0/2567160959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d18004870 0x7f4d18004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d34bdd700 1 --2- 172.21.15.3:0/2567160959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d18003a50 0x7f4d18003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d353de700 1 --2- 172.21.15.3:0/2567160959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d18005220 0x7f4d1800cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d27fff700 1 -- 172.21.15.3:0/2567160959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d18003a50 msgr2=0x7f4d18003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:44.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d27fff700 1 --2- 172.21.15.3:0/2567160959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d18003a50 0x7f4d18003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d27fff700 1 -- 172.21.15.3:0/2567160959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d18005220 msgr2=0x7f4d1800cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:44.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d27fff700 1 --2- 172.21.15.3:0/2567160959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d18005220 0x7f4d1800cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d27fff700 1 -- 172.21.15.3:0/2567160959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180035d0 con 0x7f4d18004870 2023-03-03T06:25:44.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d27fff700 1 --2- 172.21.15.3:0/2567160959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d18004870 0x7f4d18004ce0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4d3006a8e0 tx=0x7f4d180058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93e827ee8db7081 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:44.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2000c070 con 0x7f4d18004870 2023-03-03T06:25:44.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d20013630 con 0x7f4d18004870 2023-03-03T06:25:44.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.983+0000 7f4d277fe700 1 -- 172.21.15.3:0/2567160959 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d200139c0 con 0x7f4d18004870 2023-03-03T06:25:44.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.987+0000 7f4d35bdf700 1 -- 172.21.15.3:0/2567160959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d18004870 msgr2=0x7f4d18004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.987+0000 7f4d35bdf700 1 --2- 172.21.15.3:0/2567160959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d18004870 0x7f4d18004ce0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4d3006a8e0 tx=0x7f4d180058a0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.987+0000 7f4d353de700 1 -- 172.21.15.3:0/2567160959 reap_dead start 2023-03-03T06:25:44.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.987+0000 7f4d35bdf700 1 -- 172.21.15.3:0/2567160959 shutdown_connections 2023-03-03T06:25:44.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.987+0000 7f4d35bdf700 1 -- 172.21.15.3:0/2567160959 >> 172.21.15.3:0/2567160959 conn(0x7f4d30070a30 msgr2=0x7f4d30070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:44.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.987+0000 7f4d35bdf700 1 -- 172.21.15.3:0/2567160959 shutdown_connections 2023-03-03T06:25:44.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.987+0000 7f4d35bdf700 1 -- 172.21.15.3:0/2567160959 wait complete. 2023-03-03T06:25:44.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.991+0000 7f4d35bdf700 1 Processor -- start 2023-03-03T06:25:44.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.991+0000 7f4d35bdf700 1 -- start start 2023-03-03T06:25:44.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.991+0000 7f4d35bdf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d18004870 0x7f4d30157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.991+0000 7f4d35bdf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d300a74b0 0x7f4d300c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.991+0000 7f4d35bdf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d300a7920 0x7f4d300ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:44.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.991+0000 7f4d35bdf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d301e1690 con 0x7f4d300a7920 2023-03-03T06:25:44.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.991+0000 7f4d35bdf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d301e1970 con 0x7f4d18004870 2023-03-03T06:25:44.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.991+0000 7f4d35bdf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d301e1c50 con 0x7f4d300a74b0 2023-03-03T06:25:44.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.991+0000 7f4d353de700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d300a7920 0x7f4d300ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.991+0000 7f4d27fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d300a74b0 0x7f4d300c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.991+0000 7f4d353de700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d300a7920 0x7f4d300ca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43342/0 (socket says 172.21.15.3:43342) 2023-03-03T06:25:44.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.991+0000 7f4d353de700 1 -- 172.21.15.3:0/1117772341 learned_addr learned my addr 172.21.15.3:0/1117772341 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:44.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.991+0000 7f4d34bdd700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d18004870 0x7f4d30157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:44.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.995+0000 7f4d27fff700 1 -- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d18004870 msgr2=0x7f4d30157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.995+0000 7f4d27fff700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d18004870 0x7f4d30157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.995+0000 7f4d27fff700 1 -- 172.21.15.3:0/1117772341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d300a7920 msgr2=0x7f4d300ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:44.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.995+0000 7f4d27fff700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d300a7920 0x7f4d300ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:44.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.995+0000 7f4d27fff700 1 -- 172.21.15.3:0/1117772341 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d300cc860 con 0x7f4d300a74b0 2023-03-03T06:25:45.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.995+0000 7f4d353de700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d300a7920 0x7f4d300ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:45.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.995+0000 7f4d27fff700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d300a74b0 0x7f4d300c9c80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4d3006a8e0 tx=0x7f4d200047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.995+0000 7f4d257fa700 1 -- 172.21.15.3:0/1117772341 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20026480 con 0x7f4d300a74b0 2023-03-03T06:25:45.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.995+0000 7f4d257fa700 1 -- 172.21.15.3:0/1117772341 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d20021070 con 0x7f4d300a74b0 2023-03-03T06:25:45.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.995+0000 7f4d257fa700 1 -- 172.21.15.3:0/1117772341 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20026840 con 0x7f4d300a74b0 2023-03-03T06:25:45.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.995+0000 7f4d35bdf700 1 -- 172.21.15.3:0/1117772341 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d300cc580 con 0x7f4d300a74b0 2023-03-03T06:25:45.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.995+0000 7f4d35bdf700 1 -- 172.21.15.3:0/1117772341 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d300ccb40 con 0x7f4d300a74b0 2023-03-03T06:25:45.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.995+0000 7f4d35bdf700 1 -- 172.21.15.3:0/1117772341 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d30073e90 con 0x7f4d300a74b0 2023-03-03T06:25:45.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:44.999+0000 7f4d257fa700 1 -- 172.21.15.3:0/1117772341 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4d2001c420 con 0x7f4d300a74b0 2023-03-03T06:25:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.003+0000 7f4d257fa700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d1c064ee0 0x7f4d1c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.003+0000 7f4d257fa700 1 -- 172.21.15.3:0/1117772341 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4d20000e40 con 0x7f4d300a74b0 2023-03-03T06:25:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.003+0000 7f4d34bdd700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d1c064ee0 0x7f4d1c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.003+0000 7f4d34bdd700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d1c064ee0 0x7f4d1c067390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c001390 tx=0x7f4d2c007df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.007+0000 7f4d257fa700 1 -- 172.21.15.3:0/1117772341 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4d200516b0 con 0x7f4d300a74b0 2023-03-03T06:25:45.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.191+0000 7f4d35bdf700 1 -- 172.21.15.3:0/1117772341 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d300cee00 con 0x7f4d300a74b0 2023-03-03T06:25:45.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.195+0000 7f4d257fa700 1 -- 172.21.15.3:0/1117772341 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19556 (secure 0 0 0) 0x7f4d20054cd0 con 0x7f4d300a74b0 2023-03-03T06:25:45.202 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:45.202 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:18.863350+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:25:45.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.203+0000 7f4d35bdf700 1 -- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d1c064ee0 msgr2=0x7f4d1c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.203+0000 7f4d35bdf700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d1c064ee0 0x7f4d1c067390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c001390 tx=0x7f4d2c007df0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.203+0000 7f4d35bdf700 1 -- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d300a74b0 msgr2=0x7f4d300c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.203+0000 7f4d35bdf700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d300a74b0 0x7f4d300c9c80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4d3006a8e0 tx=0x7f4d200047e0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.203+0000 7f4d35bdf700 1 -- 172.21.15.3:0/1117772341 shutdown_connections 2023-03-03T06:25:45.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.203+0000 7f4d35bdf700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d1c064ee0 0x7f4d1c067390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.203+0000 7f4d35bdf700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d300a7920 0x7f4d300ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.203+0000 7f4d35bdf700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d300a74b0 0x7f4d300c9c80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.203+0000 7f4d35bdf700 1 --2- 172.21.15.3:0/1117772341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d18004870 0x7f4d30157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.203+0000 7f4d35bdf700 1 -- 172.21.15.3:0/1117772341 >> 172.21.15.3:0/1117772341 conn(0x7f4d30070a30 msgr2=0x7f4d3006d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.203+0000 7f4d35bdf700 1 -- 172.21.15.3:0/1117772341 shutdown_connections 2023-03-03T06:25:45.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.203+0000 7f4d35bdf700 1 -- 172.21.15.3:0/1117772341 wait complete. 2023-03-03T06:25:45.242 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-03-03T06:25:45.242 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-03-03T06:25:45.243 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-03-03T06:25:45.243 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-03-03T06:25:45.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-03-03T06:25:45.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-03-03T06:25:45.245 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-03-03T06:25:45.245 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-03-03T06:25:45.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.527+0000 7feb3a596700 1 Processor -- start 2023-03-03T06:25:45.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.527+0000 7feb3a596700 1 -- start start 2023-03-03T06:25:45.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.527+0000 7feb3a596700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb340c9a60 0x7feb3415e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.527+0000 7feb3a596700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb340aec00 0x7feb340af020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.527+0000 7feb3a596700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb340b04f0 con 0x7feb3415d8e0 2023-03-03T06:25:45.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.527+0000 7feb3a596700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3415ee30 con 0x7feb340c9a60 2023-03-03T06:25:45.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.527+0000 7feb3a596700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3415dcb0 con 0x7feb340aec00 2023-03-03T06:25:45.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb33fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feb3415d8e0 0x7feb340b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:56494/0 (socket says 172.21.15.3:56494) 2023-03-03T06:25:45.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb33fff700 1 -- 172.21.15.3:0/2090371382 learned_addr learned my addr 172.21.15.3:0/2090371382 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:45.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb38b33700 1 --2- 172.21.15.3:0/2090371382 >> v2:172.21.15.32:3300/0 conn(0x7feb340aec00 0x7feb340af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb337fe700 1 --2- 172.21.15.3:0/2090371382 >> v2:172.21.15.3:3300/0 conn(0x7feb340c9a60 0x7feb3415e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb32ffd700 1 -- 172.21.15.3:0/2090371382 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100180814 0 0) 0x7feb340b04f0 con 0x7feb3415d8e0 2023-03-03T06:25:45.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb32ffd700 1 -- 172.21.15.3:0/2090371382 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb1c0035d0 con 0x7feb3415d8e0 2023-03-03T06:25:45.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb38b33700 1 -- 172.21.15.3:0/2090371382 >> v2:172.21.15.3:3300/0 conn(0x7feb340c9a60 msgr2=0x7feb3415e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:45.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb38b33700 1 --2- 172.21.15.3:0/2090371382 >> v2:172.21.15.3:3300/0 conn(0x7feb340c9a60 0x7feb3415e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb38b33700 1 -- 172.21.15.3:0/2090371382 >> v1:172.21.15.32:6789/0 conn(0x7feb3415d8e0 legacy=0x7feb340b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb38b33700 1 -- 172.21.15.3:0/2090371382 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb340c9ee0 con 0x7feb340aec00 2023-03-03T06:25:45.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb337fe700 1 --2- 172.21.15.3:0/2090371382 >> v2:172.21.15.3:3300/0 conn(0x7feb340c9a60 0x7feb3415e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:45.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb38b33700 1 --2- 172.21.15.3:0/2090371382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb340aec00 0x7feb340af020 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feb3406c060 tx=0x7feb28009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=502d026e354256a2 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7effff59e700 1 Processor -- start 2023-03-03T06:25:45.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7effff59e700 1 -- start start 2023-03-03T06:25:45.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb32ffd700 1 -- 172.21.15.3:0/2090371382 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28011830 con 0x7feb340aec00 2023-03-03T06:25:45.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb32ffd700 1 -- 172.21.15.3:0/2090371382 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb28011990 con 0x7feb340aec00 2023-03-03T06:25:45.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.531+0000 7feb32ffd700 1 -- 172.21.15.3:0/2090371382 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28011d20 con 0x7feb340aec00 2023-03-03T06:25:45.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7feb3a596700 1 -- 172.21.15.3:0/2090371382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb340aec00 msgr2=0x7feb340af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7feb3a596700 1 --2- 172.21.15.3:0/2090371382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb340aec00 0x7feb340af020 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feb3406c060 tx=0x7feb28009b30 comp rx=0 tx=0).stop 2023-03-03T06:25:45.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7feb3a596700 1 -- 172.21.15.3:0/2090371382 shutdown_connections 2023-03-03T06:25:45.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7feb3a596700 1 --2- 172.21.15.3:0/2090371382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb340aec00 0x7feb340af020 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7feb3a596700 1 --2- 172.21.15.3:0/2090371382 >> v2:172.21.15.3:3300/0 conn(0x7feb340c9a60 0x7feb3415e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7feb3a596700 1 -- 172.21.15.3:0/2090371382 >> 172.21.15.3:0/2090371382 conn(0x7feb34060be0 msgr2=0x7feb34070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7feb3a596700 1 -- 172.21.15.3:0/2090371382 shutdown_connections 2023-03-03T06:25:45.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7effff59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0000158f00 0x7f00001592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7effff59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00000cde60 0x7f00000ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7effff59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00000ae710 con 0x7f00000afb60 2023-03-03T06:25:45.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7effff59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000015d8a0 con 0x7f0000158f00 2023-03-03T06:25:45.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7effff59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000015db60 con 0x7f00000cde60 2023-03-03T06:25:45.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7efffed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f00000afb60 0x7f00000ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33662/0 (socket says 172.21.15.3:33662) 2023-03-03T06:25:45.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7efffed9d700 1 -- 172.21.15.3:0/4098650427 learned_addr learned my addr 172.21.15.3:0/4098650427 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:45.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7efffdd9b700 1 --2- 172.21.15.3:0/4098650427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00000cde60 0x7f00000ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7efffe59c700 1 --2- 172.21.15.3:0/4098650427 >> v2:172.21.15.32:3300/0 conn(0x7f0000158f00 0x7f00001592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7efffdd9b700 1 -- 172.21.15.3:0/4098650427 >> v2:172.21.15.32:3300/0 conn(0x7f0000158f00 msgr2=0x7f00001592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:45.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7efffdd9b700 1 --2- 172.21.15.3:0/4098650427 >> v2:172.21.15.32:3300/0 conn(0x7f0000158f00 0x7f00001592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7efffdd9b700 1 -- 172.21.15.3:0/4098650427 >> v1:172.21.15.3:6789/0 conn(0x7f00000afb60 legacy=0x7f00000ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7efffdd9b700 1 -- 172.21.15.3:0/4098650427 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00000caa50 con 0x7f00000cde60 2023-03-03T06:25:45.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.535+0000 7feb3a596700 1 -- 172.21.15.3:0/2090371382 wait complete. 2023-03-03T06:25:45.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7feb3a596700 1 Processor -- start 2023-03-03T06:25:45.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7efffdd9b700 1 --2- 172.21.15.3:0/4098650427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00000cde60 0x7f00000ae1d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7effe8009850 tx=0x7effe8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b55146a500c7f1a server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7feb3a596700 1 -- start start 2023-03-03T06:25:45.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7efffd59a700 1 -- 172.21.15.3:0/4098650427 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe800f8a0 con 0x7f00000cde60 2023-03-03T06:25:45.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7efffd59a700 1 -- 172.21.15.3:0/4098650427 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe800fa00 con 0x7f00000cde60 2023-03-03T06:25:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7efffd59a700 1 -- 172.21.15.3:0/4098650427 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe800fd90 con 0x7f00000cde60 2023-03-03T06:25:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7effff59e700 1 -- 172.21.15.3:0/4098650427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00000cde60 msgr2=0x7f00000ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7effff59e700 1 --2- 172.21.15.3:0/4098650427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00000cde60 0x7f00000ae1d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7effe8009850 tx=0x7effe8009b60 comp rx=0 tx=0).stop 2023-03-03T06:25:45.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7fac3cabc700 1 Processor -- start 2023-03-03T06:25:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7f3d4e448700 1 Processor -- start 2023-03-03T06:25:45.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7feb3a596700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb340aec00 0x7feb340c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7effff59e700 1 -- 172.21.15.3:0/4098650427 shutdown_connections 2023-03-03T06:25:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7effff59e700 1 --2- 172.21.15.3:0/4098650427 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00000cde60 0x7f00000ae1d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7effff59e700 1 --2- 172.21.15.3:0/4098650427 >> v2:172.21.15.32:3300/0 conn(0x7f0000158f00 0x7f00001592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7effff59e700 1 -- 172.21.15.3:0/4098650427 >> 172.21.15.3:0/4098650427 conn(0x7f0000060be0 msgr2=0x7f0000070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7f3d4e448700 1 -- start start 2023-03-03T06:25:45.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7feb33fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb340aec00 0x7feb340c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7feb33fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb340aec00 0x7feb340c2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43364/0 (socket says 172.21.15.3:43364) 2023-03-03T06:25:45.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7feb33fff700 1 -- 172.21.15.3:0/2297281263 learned_addr learned my addr 172.21.15.3:0/2297281263 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:45.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7feb3a596700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb340c9a60 0x7feb341f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7feb3a596700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb3415d8e0 0x7feb341f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7f3d4e448700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d48158660 0x7f3d480b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7effff59e700 1 -- 172.21.15.3:0/4098650427 shutdown_connections 2023-03-03T06:25:45.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7f3d4e448700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d480b0c60 0x7f3d480ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.539+0000 7fac3cabc700 1 -- start start 2023-03-03T06:25:45.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7fac3cabc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac3003a670 0x7fac30051140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7fac3cabc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac30037130 0x7fac30052ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7fac3cabc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac300547b0 con 0x7fac30054a20 2023-03-03T06:25:45.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7fac3cabc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac30007320 con 0x7fac30037130 2023-03-03T06:25:45.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7fac3cabc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac30051800 con 0x7fac3003a670 2023-03-03T06:25:45.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7feb337fe700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb340c9a60 0x7feb341f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7feb3a596700 1 -- 172.21.15.3:0/2297281263 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb34157360 con 0x7feb340aec00 2023-03-03T06:25:45.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7feb38b33700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb3415d8e0 0x7feb341f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7feb3a596700 1 -- 172.21.15.3:0/2297281263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb340c22d0 con 0x7feb340c9a60 2023-03-03T06:25:45.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7f3d4e448700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d480c8e90 con 0x7f3d48060fe0 2023-03-03T06:25:45.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7f3d4e448700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d48158a30 con 0x7f3d48158660 2023-03-03T06:25:45.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7f3d4e448700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d480c9100 con 0x7f3d480b0c60 2023-03-03T06:25:45.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7fac36d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac30037130 0x7fac30052ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7feb3a596700 1 -- 172.21.15.3:0/2297281263 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb340c50d0 con 0x7feb3415d8e0 2023-03-03T06:25:45.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7fac36d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac30037130 0x7fac30052ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43372/0 (socket says 172.21.15.3:43372) 2023-03-03T06:25:45.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7fac36d9d700 1 -- 172.21.15.3:0/2980398549 learned_addr learned my addr 172.21.15.3:0/2980398549 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:45.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7f3d47fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d48158660 0x7f3d480b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7f3d47fff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3d48158660 0x7f3d480b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39708/0 (socket says 172.21.15.3:39708) 2023-03-03T06:25:45.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7f3d47fff700 1 -- 172.21.15.3:0/319700926 learned_addr learned my addr 172.21.15.3:0/319700926 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:45.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7f3d477fe700 1 --2- 172.21.15.3:0/319700926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d480b0c60 0x7f3d480ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d47fff700 1 -- 172.21.15.3:0/319700926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d480b0c60 msgr2=0x7f3d480ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:45.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d47fff700 1 --2- 172.21.15.3:0/319700926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d480b0c60 0x7f3d480ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d47fff700 1 -- 172.21.15.3:0/319700926 >> v1:172.21.15.32:6789/0 conn(0x7f3d48060fe0 legacy=0x7f3d4815c0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-03T06:25:45.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d47fff700 1 -- 172.21.15.3:0/319700926 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d480c93e0 con 0x7f3d48158660 2023-03-03T06:25:45.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7fac36d9d700 1 -- 172.21.15.3:0/2980398549 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac3003a670 msgr2=0x7fac30051140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-03T06:25:45.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7fac36d9d700 1 --2- 172.21.15.3:0/2980398549 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac3003a670 0x7fac30051140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7fac36d9d700 1 -- 172.21.15.3:0/2980398549 >> v1:172.21.15.32:6789/0 conn(0x7fac30054a20 legacy=0x7fac30036a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:45.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d47fff700 1 --2- 172.21.15.3:0/319700926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d48158660 0x7f3d480b0720 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009940 tx=0x7f3d38009c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a17de30419a6e88 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7feb33fff700 1 -- 172.21.15.3:0/2297281263 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb3415d8e0 msgr2=0x7feb341f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7feb33fff700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb3415d8e0 0x7feb341f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7feb33fff700 1 -- 172.21.15.3:0/2297281263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb340c9a60 msgr2=0x7feb341f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7feb33fff700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb340c9a60 0x7feb341f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7feb33fff700 1 -- 172.21.15.3:0/2297281263 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c009a70 con 0x7feb340aec00 2023-03-03T06:25:45.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7fac36d9d700 1 -- 172.21.15.3:0/2980398549 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3003aa40 con 0x7fac30037130 2023-03-03T06:25:45.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.543+0000 7effff59e700 1 -- 172.21.15.3:0/4098650427 wait complete. 2023-03-03T06:25:45.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d46ffd700 1 -- 172.21.15.3:0/319700926 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d38004230 con 0x7f3d48158660 2023-03-03T06:25:45.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7fac36d9d700 1 --2- 172.21.15.3:0/2980398549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac30037130 0x7fac30052ae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009b70 tx=0x7fac2c009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c6975bfedc2ea2 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7effff59e700 1 Processor -- start 2023-03-03T06:25:45.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d46ffd700 1 -- 172.21.15.3:0/319700926 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d38004390 con 0x7f3d48158660 2023-03-03T06:25:45.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d46ffd700 1 -- 172.21.15.3:0/319700926 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d380036c0 con 0x7f3d48158660 2023-03-03T06:25:45.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7effff59e700 1 -- start start 2023-03-03T06:25:45.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7feb33fff700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb340aec00 0x7feb340c2fe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feb2400c730 tx=0x7feb2400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7feb30ff9700 1 -- 172.21.15.3:0/2297281263 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24005d90 con 0x7feb340aec00 2023-03-03T06:25:45.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7feb30ff9700 1 -- 172.21.15.3:0/2297281263 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb240070d0 con 0x7feb340aec00 2023-03-03T06:25:45.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7feb30ff9700 1 -- 172.21.15.3:0/2297281263 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24007460 con 0x7feb340aec00 2023-03-03T06:25:45.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d4e448700 1 -- 172.21.15.3:0/319700926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d48158660 msgr2=0x7f3d480b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d4e448700 1 --2- 172.21.15.3:0/319700926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d48158660 0x7f3d480b0720 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009940 tx=0x7f3d38009c50 comp rx=0 tx=0).stop 2023-03-03T06:25:45.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d4e448700 1 -- 172.21.15.3:0/319700926 shutdown_connections 2023-03-03T06:25:45.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d4e448700 1 --2- 172.21.15.3:0/319700926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d480b0c60 0x7f3d480ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d4e448700 1 --2- 172.21.15.3:0/319700926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d48158660 0x7f3d480b0720 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d4e448700 1 -- 172.21.15.3:0/319700926 >> 172.21.15.3:0/319700926 conn(0x7f3d48060be0 msgr2=0x7f3d48070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7feb3a596700 1 -- 172.21.15.3:0/2297281263 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb280094b0 con 0x7feb340aec00 2023-03-03T06:25:45.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d4e448700 1 -- 172.21.15.3:0/319700926 shutdown_connections 2023-03-03T06:25:45.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7fac3559a700 1 -- 172.21.15.3:0/2980398549 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c004230 con 0x7fac30037130 2023-03-03T06:25:45.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.547+0000 7f3d4e448700 1 -- 172.21.15.3:0/319700926 wait complete. 2023-03-03T06:25:45.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7effff59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00000afb60 0x7f0000157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7effff59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00000cde60 0x7f00001ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7effff59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0000158f00 0x7f00001592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7effff59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001f0190 con 0x7f0000158f00 2023-03-03T06:25:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7efffdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00000cde60 0x7f00001ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7efffe59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00000afb60 0x7f0000157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7efffdd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00000cde60 0x7f00001ef6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39720/0 (socket says 172.21.15.3:39720) 2023-03-03T06:25:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7efffdd9b700 1 -- 172.21.15.3:0/2462062034 learned_addr learned my addr 172.21.15.3:0/2462062034 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:45.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7efffed9d700 1 --2- 172.21.15.3:0/2462062034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0000158f00 0x7f00001592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7fac3559a700 1 -- 172.21.15.3:0/2980398549 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2c004390 con 0x7fac30037130 2023-03-03T06:25:45.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7f3d4e448700 1 Processor -- start 2023-03-03T06:25:45.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7fac3559a700 1 -- 172.21.15.3:0/2980398549 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c00b6d0 con 0x7fac30037130 2023-03-03T06:25:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7effff59e700 1 -- 172.21.15.3:0/2462062034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001f0470 con 0x7f00000cde60 2023-03-03T06:25:45.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7effff59e700 1 -- 172.21.15.3:0/2462062034 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00001f0750 con 0x7f00000afb60 2023-03-03T06:25:45.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7f3d4e448700 1 -- start start 2023-03-03T06:25:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7fac3cabc700 1 -- 172.21.15.3:0/2980398549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac30037130 msgr2=0x7fac30052ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7fac3cabc700 1 --2- 172.21.15.3:0/2980398549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac30037130 0x7fac30052ae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009b70 tx=0x7fac2c009e80 comp rx=0 tx=0).stop 2023-03-03T06:25:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7efffe59c700 1 -- 172.21.15.3:0/2462062034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00000cde60 msgr2=0x7f00001ef6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7efffe59c700 1 --2- 172.21.15.3:0/2462062034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00000cde60 0x7f00001ef6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7efffe59c700 1 -- 172.21.15.3:0/2462062034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0000158f00 msgr2=0x7f00001592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7efffe59c700 1 --2- 172.21.15.3:0/2462062034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0000158f00 0x7f00001592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7efffe59c700 1 -- 172.21.15.3:0/2462062034 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe80094b0 con 0x7f00000afb60 2023-03-03T06:25:45.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7efffe59c700 1 --2- 172.21.15.3:0/2462062034 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00000afb60 0x7f0000157bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efff000e810 tx=0x7efff000eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7fac3cabc700 1 -- 172.21.15.3:0/2980398549 shutdown_connections 2023-03-03T06:25:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7fac3cabc700 1 --2- 172.21.15.3:0/2980398549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac30037130 0x7fac30052ae0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7fac3cabc700 1 --2- 172.21.15.3:0/2980398549 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac3003a670 0x7fac30051140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7fac3cabc700 1 -- 172.21.15.3:0/2980398549 >> 172.21.15.3:0/2980398549 conn(0x7fac30051a20 msgr2=0x7fac30051e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7efff6ffd700 1 -- 172.21.15.3:0/2462062034 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff000c070 con 0x7f00000afb60 2023-03-03T06:25:45.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7efff6ffd700 1 -- 172.21.15.3:0/2462062034 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff0013610 con 0x7f00000afb60 2023-03-03T06:25:45.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7efff6ffd700 1 -- 172.21.15.3:0/2462062034 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff00139c0 con 0x7f00000afb60 2023-03-03T06:25:45.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7effff59e700 1 -- 172.21.15.3:0/2462062034 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00000cec60 con 0x7f00000afb60 2023-03-03T06:25:45.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d4e448700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d48060fe0 0x7f3d48156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d4e448700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480b0c60 0x7f3d480b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d4e448700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d48158660 0x7f3d480b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d4e448700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d480b2f30 con 0x7f3d480b0c60 2023-03-03T06:25:45.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d4e448700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d480b33c0 con 0x7f3d48158660 2023-03-03T06:25:45.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d4e448700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d480b2320 con 0x7f3d48060fe0 2023-03-03T06:25:45.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d477fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480b0c60 0x7f3d480b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d477fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480b0c60 0x7f3d480b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43392/0 (socket says 172.21.15.3:43392) 2023-03-03T06:25:45.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d477fe700 1 -- 172.21.15.3:0/898193348 learned_addr learned my addr 172.21.15.3:0/898193348 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:45.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7fac3cabc700 1 -- 172.21.15.3:0/2980398549 shutdown_connections 2023-03-03T06:25:45.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d47fff700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d48060fe0 0x7f3d48156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d4c9e5700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d48158660 0x7f3d480b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7fac3cabc700 1 -- 172.21.15.3:0/2980398549 wait complete. 2023-03-03T06:25:45.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7effff59e700 1 -- 172.21.15.3:0/2462062034 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00000cad30 con 0x7f00000afb60 2023-03-03T06:25:45.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.551+0000 7feb30ff9700 1 -- 172.21.15.3:0/2297281263 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7feb24019030 con 0x7feb340aec00 2023-03-03T06:25:45.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7feb30ff9700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7feb20064f60 0x7feb20067410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7feb337fe700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7feb20064f60 0x7feb20067410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d477fe700 1 -- 172.21.15.3:0/898193348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d48060fe0 msgr2=0x7f3d48156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d477fe700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d48060fe0 0x7f3d48156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d477fe700 1 -- 172.21.15.3:0/898193348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d48158660 msgr2=0x7f3d480b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d477fe700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d48158660 0x7f3d480b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d477fe700 1 -- 172.21.15.3:0/898193348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d38009570 con 0x7f3d480b0c60 2023-03-03T06:25:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7f3d47fff700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3d48060fe0 0x7f3d48156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7f3d477fe700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480b0c60 0x7f3d480b3d50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00e9d0 tx=0x7f3d3c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7effff59e700 1 -- 172.21.15.3:0/2462062034 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f00000cedf0 con 0x7f00000afb60 2023-03-03T06:25:45.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.555+0000 7feb3a596700 1 -- 172.21.15.3:0/2297281263 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb340ca140 con 0x7feb340aec00 2023-03-03T06:25:45.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7feb30ff9700 1 -- 172.21.15.3:0/2297281263 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7feb2405ae40 con 0x7feb340aec00 2023-03-03T06:25:45.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7feb337fe700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7feb20064f60 0x7feb20067410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009fd0 tx=0x7feb2c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7f3d44ff9700 1 -- 172.21.15.3:0/898193348 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c009720 con 0x7f3d480b0c60 2023-03-03T06:25:45.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7f3d4e448700 1 -- 172.21.15.3:0/898193348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d480b1170 con 0x7f3d480b0c60 2023-03-03T06:25:45.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7f4aec25b700 1 Processor -- start 2023-03-03T06:25:45.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7fac3cabc700 1 Processor -- start 2023-03-03T06:25:45.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7f4aec25b700 1 -- start start 2023-03-03T06:25:45.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7f3d4e448700 1 -- 172.21.15.3:0/898193348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d480c96c0 con 0x7f3d480b0c60 2023-03-03T06:25:45.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7fac3cabc700 1 -- start start 2023-03-03T06:25:45.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7fdeb89e6700 1 Processor -- start 2023-03-03T06:25:45.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7efff6ffd700 1 -- 172.21.15.3:0/2462062034 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7efff0023070 con 0x7f00000afb60 2023-03-03T06:25:45.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7fdeb89e6700 1 -- start start 2023-03-03T06:25:45.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7fac3cabc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac30037130 0x7fac300e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7f3d4e448700 1 -- 172.21.15.3:0/898193348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d480ae100 con 0x7f3d480b0c60 2023-03-03T06:25:45.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7f3d44ff9700 1 -- 172.21.15.3:0/898193348 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c00c850 con 0x7f3d480b0c60 2023-03-03T06:25:45.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7feb3a596700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.3:6808/32272,v1:172.21.15.3:6809/32272] conn(0x7feb3415d0c0 0x7feb340c2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7feb3a596700 1 -- 172.21.15.3:0/2297281263 --> [v2:172.21.15.3:6808/32272,v1:172.21.15.3:6809/32272] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb340af560 con 0x7feb3415d0c0 2023-03-03T06:25:45.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.559+0000 7feb38b33700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.3:6808/32272,v1:172.21.15.3:6809/32272] conn(0x7feb3415d0c0 0x7feb340c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f3d44ff9700 1 -- 172.21.15.3:0/898193348 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c00cbc0 con 0x7f3d480b0c60 2023-03-03T06:25:45.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac3659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac30037130 0x7fac300e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac3659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac30037130 0x7fac300e92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43394/0 (socket says 172.21.15.3:43394) 2023-03-03T06:25:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac3659c700 1 -- 172.21.15.3:0/3334343514 learned_addr learned my addr 172.21.15.3:0/3334343514 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:45.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac3cabc700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac3003a670 0x7fac3017f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac3cabc700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac30054a20 0x7fac30038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac3cabc700 1 -- 172.21.15.3:0/3334343514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac300e8230 con 0x7fac30037130 2023-03-03T06:25:45.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7efff6ffd700 1 --2- 172.21.15.3:0/2462062034 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7effe0064f30 0x7effe00673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7efff6ffd700 1 -- 172.21.15.3:0/2462062034 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efff0081d50 con 0x7f00000afb60 2023-03-03T06:25:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7efff6ffd700 1 --2- 172.21.15.3:0/2462062034 >> [v2:172.21.15.3:6824/32269,v1:172.21.15.3:6825/32269] conn(0x7effe006ec00 0x7effe0071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7efff6ffd700 1 -- 172.21.15.3:0/2462062034 --> [v2:172.21.15.3:6824/32269,v1:172.21.15.3:6825/32269] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7effe0071700 con 0x7effe006ec00 2023-03-03T06:25:45.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7efff6ffd700 1 -- 172.21.15.3:0/2462062034 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efff0084440 con 0x7f00000afb60 2023-03-03T06:25:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7efffdd9b700 1 --2- 172.21.15.3:0/2462062034 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7effe0064f30 0x7effe00673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac3cabc700 1 -- 172.21.15.3:0/3334343514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac300e8510 con 0x7fac3003a670 2023-03-03T06:25:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac3cabc700 1 -- 172.21.15.3:0/3334343514 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac300e87f0 con 0x7fac30054a20 2023-03-03T06:25:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac36d9d700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac30054a20 0x7fac30038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac35d9b700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac3003a670 0x7fac3017f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f4aec25b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ae41590b0 0x7f4ae406cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f4aec25b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ae40ceec0 0x7f4ae40b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f4aec25b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae4159480 con 0x7f4ae40b1740 2023-03-03T06:25:45.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f4aec25b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae406d6b0 con 0x7f4ae40ceec0 2023-03-03T06:25:45.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f4aec25b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae406d3d0 con 0x7f4ae41590b0 2023-03-03T06:25:45.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f4ae9ff7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ae41590b0 0x7f4ae406cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f4ae97f6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4ae40b1740 0x7f4ae40ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:56514/0 (socket says 172.21.15.3:56514) 2023-03-03T06:25:45.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f4ae97f6700 1 -- 172.21.15.3:0/2036604046 learned_addr learned my addr 172.21.15.3:0/2036604046 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:45.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f4aeaff9700 1 --2- 172.21.15.3:0/2036604046 >> v2:172.21.15.32:3300/0 conn(0x7f4ae40ceec0 0x7f4ae40b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb89e6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdeb4160880 0x7fdeb40c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb89e6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdeb40aea70 0x7fdeb40ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb89e6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeb40adcf0 0x7fdeb41644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb89e6700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb40ae190 con 0x7fdeb4160880 2023-03-03T06:25:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb89e6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb40ce190 con 0x7fdeb40aea70 2023-03-03T06:25:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb89e6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb40c99d0 con 0x7fdeb40adcf0 2023-03-03T06:25:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdeb4160880 0x7fdeb40c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb259c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdeb4160880 0x7fdeb40c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43400/0 (socket says 172.21.15.3:43400) 2023-03-03T06:25:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb259c700 1 -- 172.21.15.3:0/2935259523 learned_addr learned my addr 172.21.15.3:0/2935259523 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:45.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7efffed9d700 1 --2- 172.21.15.3:0/2462062034 >> [v2:172.21.15.3:6824/32269,v1:172.21.15.3:6825/32269] conn(0x7effe006ec00 0x7effe0071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac36d9d700 1 -- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac3003a670 msgr2=0x7fac3017f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac36d9d700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac3003a670 0x7fac3017f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac36d9d700 1 -- 172.21.15.3:0/3334343514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac30037130 msgr2=0x7fac300e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac36d9d700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac30037130 0x7fac300e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac36d9d700 1 -- 172.21.15.3:0/3334343514 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c009570 con 0x7fac30054a20 2023-03-03T06:25:45.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb2d9d700 1 --2- 172.21.15.3:0/2935259523 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeb40adcf0 0x7fdeb41644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7feb38b33700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.3:6808/32272,v1:172.21.15.3:6809/32272] conn(0x7feb3415d0c0 0x7feb340c2410 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac3659c700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac30037130 0x7fac300e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb259c700 1 -- 172.21.15.3:0/2935259523 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeb40adcf0 msgr2=0x7fdeb41644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb259c700 1 --2- 172.21.15.3:0/2935259523 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeb40adcf0 0x7fdeb41644f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb259c700 1 -- 172.21.15.3:0/2935259523 >> v2:172.21.15.32:3300/0 conn(0x7fdeb40aea70 msgr2=0x7fdeb40ad7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-03T06:25:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb259c700 1 --2- 172.21.15.3:0/2935259523 >> v2:172.21.15.32:3300/0 conn(0x7fdeb40aea70 0x7fdeb40ad7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fdeb259c700 1 -- 172.21.15.3:0/2935259523 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb40cd580 con 0x7fdeb4160880 2023-03-03T06:25:45.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f4aeaff9700 1 -- 172.21.15.3:0/2036604046 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ae41590b0 msgr2=0x7f4ae406cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:45.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f4aeaff9700 1 --2- 172.21.15.3:0/2036604046 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ae41590b0 0x7f4ae406cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f4aeaff9700 1 -- 172.21.15.3:0/2036604046 >> v1:172.21.15.32:6789/0 conn(0x7f4ae40b1740 legacy=0x7f4ae40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7f4aeaff9700 1 -- 172.21.15.3:0/2036604046 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae406e760 con 0x7f4ae40ceec0 2023-03-03T06:25:45.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7feb30ff9700 1 -- 172.21.15.3:0/2297281263 <== osd.1 v2:172.21.15.3:6808/32272 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7feb340af560 con 0x7feb3415d0c0 2023-03-03T06:25:45.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.563+0000 7fac36d9d700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac30054a20 0x7fac30038e20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac2c015000 tx=0x7fac2c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7efffdd9b700 1 --2- 172.21.15.3:0/2462062034 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7effe0064f30 0x7effe00673e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f000006dd80 tx=0x7effe8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7efffed9d700 1 --2- 172.21.15.3:0/2462062034 >> [v2:172.21.15.3:6824/32269,v1:172.21.15.3:6825/32269] conn(0x7effe006ec00 0x7effe0071030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f3d44ff9700 1 -- 172.21.15.3:0/898193348 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3d3c019030 con 0x7f3d480b0c60 2023-03-03T06:25:45.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f2772550700 1 Processor -- start 2023-03-03T06:25:45.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7fdeb259c700 1 --2- 172.21.15.3:0/2935259523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb4160880 0x7fdeb40c9dc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40cd160 tx=0x7fdeac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b51b3fbf4ba44f server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f2772550700 1 -- start start 2023-03-03T06:25:45.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7fac26ffd700 1 -- 172.21.15.3:0/3334343514 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c00b6d0 con 0x7fac30054a20 2023-03-03T06:25:45.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7fac26ffd700 1 -- 172.21.15.3:0/3334343514 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2c00b830 con 0x7fac30054a20 2023-03-03T06:25:45.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7fac26ffd700 1 -- 172.21.15.3:0/3334343514 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c00bbe0 con 0x7fac30054a20 2023-03-03T06:25:45.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7efff6ffd700 1 -- 172.21.15.3:0/2462062034 <== osd.0 v2:172.21.15.3:6824/32269 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7effe0071700 con 0x7effe006ec00 2023-03-03T06:25:45.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7fac3cabc700 1 -- 172.21.15.3:0/3334343514 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac30007040 con 0x7fac30054a20 2023-03-03T06:25:45.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7fac3cabc700 1 -- 172.21.15.3:0/3334343514 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac30006d60 con 0x7fac30054a20 2023-03-03T06:25:45.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f3d44ff9700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3d34064fb0 0x7f3d34067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f3d44ff9700 1 -- 172.21.15.3:0/898193348 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3d3c082540 con 0x7f3d480b0c60 2023-03-03T06:25:45.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7fac24ff9700 1 -- 172.21.15.3:0/3334343514 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac10000fc0 con 0x7fac30054a20 2023-03-03T06:25:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f4aeaff9700 1 --2- 172.21.15.3:0/2036604046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ae40ceec0 0x7f4ae40b3090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0025e0 tx=0x7f4adc007db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b395cde491ac36f3 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7fdeb159a700 1 -- 172.21.15.3:0/2935259523 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00e070 con 0x7fdeb4160880 2023-03-03T06:25:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7fdeb159a700 1 -- 172.21.15.3:0/2935259523 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac00b9c0 con 0x7fdeb4160880 2023-03-03T06:25:45.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7fdeb159a700 1 -- 172.21.15.3:0/2935259523 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00bd70 con 0x7fdeb4160880 2023-03-03T06:25:45.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f3d44ff9700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.3:6800/32271,v1:172.21.15.3:6801/32271] conn(0x7f3d3406eb10 0x7f3d34070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f3d47fff700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3d34064fb0 0x7f3d34067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f3d44ff9700 1 -- 172.21.15.3:0/898193348 --> [v2:172.21.15.3:6800/32271,v1:172.21.15.3:6801/32271] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d34071610 con 0x7f3d3406eb10 2023-03-03T06:25:45.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f3d44ff9700 1 -- 172.21.15.3:0/898193348 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d3c0828f0 con 0x7f3d480b0c60 2023-03-03T06:25:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7fdeb89e6700 1 -- 172.21.15.3:0/2935259523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb4160880 msgr2=0x7fdeb40c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7fdeb89e6700 1 --2- 172.21.15.3:0/2935259523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb4160880 0x7fdeb40c9dc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40cd160 tx=0x7fdeac00b330 comp rx=0 tx=0).stop 2023-03-03T06:25:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f2772550700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f276403e9a0 0x7f2764055260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f2772550700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f276403b240 0x7f2764056c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f2772550700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2764053df0 con 0x7f2764058b60 2023-03-03T06:25:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f2772550700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2764053830 con 0x7f276403b240 2023-03-03T06:25:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.567+0000 7f2772550700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2764053b10 con 0x7f276403e9a0 2023-03-03T06:25:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7fdeb89e6700 1 -- 172.21.15.3:0/2935259523 shutdown_connections 2023-03-03T06:25:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7fdeb89e6700 1 --2- 172.21.15.3:0/2935259523 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeb40adcf0 0x7fdeb41644f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7fdeb89e6700 1 --2- 172.21.15.3:0/2935259523 >> v2:172.21.15.32:3300/0 conn(0x7fdeb40aea70 0x7fdeb40ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7fdeb89e6700 1 --2- 172.21.15.3:0/2935259523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb4160880 0x7fdeb40c9dc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7fdeb89e6700 1 -- 172.21.15.3:0/2935259523 >> 172.21.15.3:0/2935259523 conn(0x7fdeb4070a30 msgr2=0x7fdeb4070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7fac26ffd700 1 -- 172.21.15.3:0/3334343514 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fac2c017020 con 0x7fac30054a20 2023-03-03T06:25:45.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f4ae8ff5700 1 -- 172.21.15.3:0/2036604046 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc00dd10 con 0x7f4ae40ceec0 2023-03-03T06:25:45.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f4ae8ff5700 1 -- 172.21.15.3:0/2036604046 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4adc00de70 con 0x7f4ae40ceec0 2023-03-03T06:25:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f4ae8ff5700 1 -- 172.21.15.3:0/2036604046 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc017670 con 0x7f4ae40ceec0 2023-03-03T06:25:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f276b7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2764058b60 0x7f276403ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:56520/0 (socket says 172.21.15.3:56520) 2023-03-03T06:25:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f276b7fe700 1 -- 172.21.15.3:0/2087801207 learned_addr learned my addr 172.21.15.3:0/2087801207 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f2770aed700 1 --2- 172.21.15.3:0/2087801207 >> v2:172.21.15.32:3300/0 conn(0x7f276403b240 0x7f2764056c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f276affd700 1 -- 172.21.15.3:0/2087801207 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567687352 0 0) 0x7f2764053df0 con 0x7f2764058b60 2023-03-03T06:25:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f276affd700 1 -- 172.21.15.3:0/2087801207 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27540035d0 con 0x7f2764058b60 2023-03-03T06:25:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f2770aed700 1 -- 172.21.15.3:0/2087801207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f276403e9a0 msgr2=0x7f2764055260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f2770aed700 1 --2- 172.21.15.3:0/2087801207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f276403e9a0 0x7f2764055260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f2770aed700 1 -- 172.21.15.3:0/2087801207 >> v1:172.21.15.32:6789/0 conn(0x7f2764058b60 legacy=0x7f276403ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f2770aed700 1 -- 172.21.15.3:0/2087801207 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2764006e40 con 0x7f276403b240 2023-03-03T06:25:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f3d4c9e5700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.3:6800/32271,v1:172.21.15.3:6801/32271] conn(0x7f3d3406eb10 0x7f3d34070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f3d47fff700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3d34064fb0 0x7f3d34067460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009940 tx=0x7f3d38008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f3d4c9e5700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.3:6800/32271,v1:172.21.15.3:6801/32271] conn(0x7f3d3406eb10 0x7f3d34070f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f4aec25b700 1 -- 172.21.15.3:0/2036604046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ae40ceec0 msgr2=0x7f4ae40b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f4aec25b700 1 --2- 172.21.15.3:0/2036604046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ae40ceec0 0x7f4ae40b3090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0025e0 tx=0x7f4adc007db0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7fdeb89e6700 1 -- 172.21.15.3:0/2935259523 shutdown_connections 2023-03-03T06:25:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7fdeb89e6700 1 -- 172.21.15.3:0/2935259523 wait complete. 2023-03-03T06:25:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f3d44ff9700 1 -- 172.21.15.3:0/898193348 <== osd.2 v2:172.21.15.3:6800/32271 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3d34071610 con 0x7f3d3406eb10 2023-03-03T06:25:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7f2770aed700 1 --2- 172.21.15.3:0/2087801207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276403b240 0x7f2764056c80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f276c04eeb0 tx=0x7f276c04e8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbdf84e2efa004b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.571+0000 7fac26ffd700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fac18064f30 0x7fac180673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7f4aec25b700 1 -- 172.21.15.3:0/2036604046 shutdown_connections 2023-03-03T06:25:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7f4aec25b700 1 --2- 172.21.15.3:0/2036604046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ae40ceec0 0x7f4ae40b3090 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7f4aec25b700 1 --2- 172.21.15.3:0/2036604046 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ae41590b0 0x7f4ae406cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7f4aec25b700 1 -- 172.21.15.3:0/2036604046 >> 172.21.15.3:0/2036604046 conn(0x7f4ae406a110 msgr2=0x7f4ae406a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7f276affd700 1 -- 172.21.15.3:0/2087801207 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c0720c0 con 0x7f276403b240 2023-03-03T06:25:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7fac26ffd700 1 -- 172.21.15.3:0/3334343514 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fac2c083f10 con 0x7fac30054a20 2023-03-03T06:25:45.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7fac26ffd700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.3:6816/32268,v1:172.21.15.3:6817/32268] conn(0x7fac1806ec00 0x7fac18071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7fac26ffd700 1 -- 172.21.15.3:0/3334343514 --> [v2:172.21.15.3:6816/32268,v1:172.21.15.3:6817/32268] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac18071700 con 0x7fac1806ec00 2023-03-03T06:25:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7fac26ffd700 1 -- 172.21.15.3:0/3334343514 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac2c084280 con 0x7fac30054a20 2023-03-03T06:25:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7f276affd700 1 -- 172.21.15.3:0/2087801207 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c072240 con 0x7f276403b240 2023-03-03T06:25:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7fac3659c700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fac18064f30 0x7fac180673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7fa7a759e700 1 Processor -- start 2023-03-03T06:25:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7fa7a759e700 1 -- start start 2023-03-03T06:25:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7f2772550700 1 -- 172.21.15.3:0/2087801207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276403b240 msgr2=0x7f2764056c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7f2772550700 1 --2- 172.21.15.3:0/2087801207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276403b240 0x7f2764056c80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f276c04eeb0 tx=0x7f276c04e8b0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7fac35d9b700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.3:6816/32268,v1:172.21.15.3:6817/32268] conn(0x7fac1806ec00 0x7fac18071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7f4aec25b700 1 -- 172.21.15.3:0/2036604046 shutdown_connections 2023-03-03T06:25:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7fac3659c700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fac18064f30 0x7fac180673e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fac38069680 tx=0x7fac38072040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7fdeb89e6700 1 Processor -- start 2023-03-03T06:25:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7fdeb89e6700 1 -- start start 2023-03-03T06:25:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7fac35d9b700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.3:6816/32268,v1:172.21.15.3:6817/32268] conn(0x7fac1806ec00 0x7fac18071030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.575+0000 7f4aec25b700 1 -- 172.21.15.3:0/2036604046 wait complete. 2023-03-03T06:25:45.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb89e6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeb40adcf0 0x7fdeb41635f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fa7a759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7a00e6f20 0x7fa7a004f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fac26ffd700 1 -- 172.21.15.3:0/3334343514 <== osd.3 v2:172.21.15.3:6816/32268 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fac18071700 con 0x7fac1806ec00 2023-03-03T06:25:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb89e6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb40aea70 0x7fdeb40c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb89e6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb4160880 0x7fdeb40b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb89e6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4163bd0 con 0x7fdeb4160880 2023-03-03T06:25:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb89e6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb40c8b20 con 0x7fdeb40aea70 2023-03-03T06:25:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb89e6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb40c8560 con 0x7fdeb40adcf0 2023-03-03T06:25:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb259c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeb40adcf0 0x7fdeb41635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fa7a759e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7a0047c50 0x7fa7a00480a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7f4aec25b700 1 Processor -- start 2023-03-03T06:25:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb4160880 0x7fdeb40b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb40aea70 0x7fdeb40c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb4160880 0x7fdeb40b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43404/0 (socket says 172.21.15.3:43404) 2023-03-03T06:25:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb2d9d700 1 -- 172.21.15.3:0/624479666 learned_addr learned my addr 172.21.15.3:0/624479666 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7f2772550700 1 -- 172.21.15.3:0/2087801207 shutdown_connections 2023-03-03T06:25:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7f2772550700 1 --2- 172.21.15.3:0/2087801207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276403b240 0x7f2764056c80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7f2772550700 1 --2- 172.21.15.3:0/2087801207 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f276403e9a0 0x7f2764055260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7f2772550700 1 -- 172.21.15.3:0/2087801207 >> 172.21.15.3:0/2087801207 conn(0x7f276400f300 msgr2=0x7f2764055b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fa7a759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a00485e0 con 0x7fa7a0005fa0 2023-03-03T06:25:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb1d9b700 1 -- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeb40adcf0 msgr2=0x7fdeb41635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb1d9b700 1 --2- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdeb40adcf0 0x7fdeb41635f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb1d9b700 1 -- 172.21.15.3:0/624479666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb4160880 msgr2=0x7fdeb40b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb1d9b700 1 --2- 172.21.15.3:0/624479666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdeb4160880 0x7fdeb40b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb1d9b700 1 -- 172.21.15.3:0/624479666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac009b30 con 0x7fdeb40aea70 2023-03-03T06:25:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdeb1d9b700 1 --2- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb40aea70 0x7fdeb40c7a30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009fd0 tx=0x7fdea400b6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fa7a6d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa7a0005fa0 0x7fa7a0006420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:56530/0 (socket says 172.21.15.3:56530) 2023-03-03T06:25:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fa7a6d9d700 1 -- 172.21.15.3:0/1344247360 learned_addr learned my addr 172.21.15.3:0/1344247360 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:45.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7f4aec25b700 1 -- start start 2023-03-03T06:25:45.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7f4aec25b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ae40b1740 0x7f4ae40b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fa7a5d9b700 1 --2- 172.21.15.3:0/1344247360 >> v2:172.21.15.3:3300/0 conn(0x7fa7a0047c50 0x7fa7a00480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7f4aec25b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ae40ceec0 0x7f4ae414bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdea2ffd700 1 -- 172.21.15.3:0/624479666 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4011830 con 0x7fdeb40aea70 2023-03-03T06:25:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdea2ffd700 1 -- 172.21.15.3:0/624479666 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea4011990 con 0x7fdeb40aea70 2023-03-03T06:25:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fdea2ffd700 1 -- 172.21.15.3:0/624479666 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4011d00 con 0x7fdeb40aea70 2023-03-03T06:25:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7f4aec25b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ae41590b0 0x7f4ae40c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fa7a559a700 1 -- 172.21.15.3:0/1344247360 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231650501 0 0) 0x7fa7a80729b0 con 0x7fa7a0005fa0 2023-03-03T06:25:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4aec25b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae4150cd0 con 0x7f4ae41590b0 2023-03-03T06:25:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4aec25b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae4150fb0 con 0x7f4ae40b1740 2023-03-03T06:25:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4aec25b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae4151290 con 0x7f4ae40ceec0 2023-03-03T06:25:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7fa7a759e700 1 -- 172.21.15.3:0/1344247360 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a0053610 con 0x7fa7a0047c50 2023-03-03T06:25:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7fa7a759e700 1 -- 172.21.15.3:0/1344247360 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0052a00 con 0x7fa7a00e6f20 2023-03-03T06:25:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7f4ae9ff7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ae40b1740 0x7f4ae40b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4ae9ff7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ae40b1740 0x7f4ae40b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39780/0 (socket says 172.21.15.3:39780) 2023-03-03T06:25:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4ae9ff7700 1 -- 172.21.15.3:0/3877249627 learned_addr learned my addr 172.21.15.3:0/3877249627 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.579+0000 7f4ae97f6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ae40ceec0 0x7f4ae414bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4aeaff9700 1 --2- 172.21.15.3:0/3877249627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ae41590b0 0x7f4ae40c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7fa7a559a700 1 -- 172.21.15.3:0/1344247360 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7900035d0 con 0x7fa7a0005fa0 2023-03-03T06:25:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f2772550700 1 -- 172.21.15.3:0/2087801207 shutdown_connections 2023-03-03T06:25:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f2772550700 1 -- 172.21.15.3:0/2087801207 wait complete. 2023-03-03T06:25:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4ae9ff7700 1 -- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ae40ceec0 msgr2=0x7f4ae414bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4ae9ff7700 1 --2- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ae40ceec0 0x7f4ae414bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4ae9ff7700 1 -- 172.21.15.3:0/3877249627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ae41590b0 msgr2=0x7f4ae40c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4ae9ff7700 1 --2- 172.21.15.3:0/3877249627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ae41590b0 0x7f4ae40c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4ae9ff7700 1 -- 172.21.15.3:0/3877249627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc007730 con 0x7f4ae40b1740 2023-03-03T06:25:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7fa7a5d9b700 1 -- 172.21.15.3:0/1344247360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7a00e6f20 msgr2=0x7fa7a004f2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-03T06:25:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7fa7a5d9b700 1 --2- 172.21.15.3:0/1344247360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7a00e6f20 0x7fa7a004f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7fa7a5d9b700 1 -- 172.21.15.3:0/1344247360 >> v1:172.21.15.32:6789/0 conn(0x7fa7a0005fa0 legacy=0x7fa7a0006420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7fa7a5d9b700 1 -- 172.21.15.3:0/1344247360 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a0052ce0 con 0x7fa7a0047c50 2023-03-03T06:25:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4ae9ff7700 1 --2- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ae40b1740 0x7f4ae40b0130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400bed0 tx=0x7f4ad400bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4ada7fc700 1 -- 172.21.15.3:0/3877249627 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400dbc0 con 0x7f4ae40b1740 2023-03-03T06:25:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7fa7a5d9b700 1 --2- 172.21.15.3:0/1344247360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7a0047c50 0x7fa7a00480a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009850 tx=0x7fa79c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34df5a999427aca server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4aec25b700 1 -- 172.21.15.3:0/3877249627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae406ed20 con 0x7f4ae40b1740 2023-03-03T06:25:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4aec25b700 1 -- 172.21.15.3:0/3877249627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae406ea40 con 0x7f4ae40b1740 2023-03-03T06:25:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7fa7a559a700 1 -- 172.21.15.3:0/1344247360 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c011830 con 0x7fa7a0047c50 2023-03-03T06:25:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7fa7a559a700 1 -- 172.21.15.3:0/1344247360 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79c011990 con 0x7fa7a0047c50 2023-03-03T06:25:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7fa7a559a700 1 -- 172.21.15.3:0/1344247360 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c011d40 con 0x7fa7a0047c50 2023-03-03T06:25:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f2772550700 1 Processor -- start 2023-03-03T06:25:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4ada7fc700 1 -- 172.21.15.3:0/3877249627 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad400f7a0 con 0x7f4ae40b1740 2023-03-03T06:25:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.583+0000 7f4ada7fc700 1 -- 172.21.15.3:0/3877249627 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400fb10 con 0x7f4ae40b1740 2023-03-03T06:25:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7fa7a759e700 1 -- 172.21.15.3:0/1344247360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7a0047c50 msgr2=0x7fa7a00480a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7fa7a759e700 1 --2- 172.21.15.3:0/1344247360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7a0047c50 0x7fa7a00480a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009850 tx=0x7fa79c009b60 comp rx=0 tx=0).stop 2023-03-03T06:25:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7f4aec25b700 1 -- 172.21.15.3:0/3877249627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4acc000fc0 con 0x7f4ae40b1740 2023-03-03T06:25:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f4ada7fc700 1 -- 172.21.15.3:0/3877249627 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4ad401d030 con 0x7f4ae40b1740 2023-03-03T06:25:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f4ada7fc700 1 --2- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4ac8064ee0 0x7f4ac8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7fdeb89e6700 1 -- 172.21.15.3:0/624479666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb40cdb40 con 0x7fdeb40aea70 2023-03-03T06:25:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7fdeb89e6700 1 -- 172.21.15.3:0/624479666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb40cd860 con 0x7fdeb40aea70 2023-03-03T06:25:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7fdea2ffd700 1 -- 172.21.15.3:0/624479666 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fdea4021070 con 0x7fdeb40aea70 2023-03-03T06:25:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7f2772550700 1 -- start start 2023-03-03T06:25:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7f2772550700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f276403b240 0x7f27640de710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7f2772550700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276403e9a0 0x7f2764050b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7f2772550700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2764058b60 0x7f276400a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7f276b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276403e9a0 0x7f2764050b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7f276b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276403e9a0 0x7f2764050b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39782/0 (socket says 172.21.15.3:39782) 2023-03-03T06:25:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7f2770aed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2764058b60 0x7f276400a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7f2770aed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2764058b60 0x7f276400a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43420/0 (socket says 172.21.15.3:43420) 2023-03-03T06:25:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.587+0000 7f276b7fe700 1 -- 172.21.15.3:0/870111000 learned_addr learned my addr 172.21.15.3:0/870111000 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2764051f10 con 0x7f2764058b60 2023-03-03T06:25:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27640dec50 con 0x7f276403e9a0 2023-03-03T06:25:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27640e4c40 con 0x7f276403b240 2023-03-03T06:25:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f2770aed700 1 -- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f276403b240 msgr2=0x7f27640de710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:25:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f2770aed700 1 --2- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f276403b240 0x7f27640de710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f2770aed700 1 -- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276403e9a0 msgr2=0x7f2764050b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f2770aed700 1 --2- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f276403e9a0 0x7f2764050b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f2770aed700 1 -- 172.21.15.3:0/870111000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2764058830 con 0x7f2764058b60 2023-03-03T06:25:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f2770aed700 1 --2- 172.21.15.3:0/870111000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2764058b60 0x7f276400a390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f276c073980 tx=0x7f276c069c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f2768ff9700 1 -- 172.21.15.3:0/870111000 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c067f20 con 0x7f2764058b60 2023-03-03T06:25:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f4ada7fc700 1 -- 172.21.15.3:0/3877249627 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4ad40818a0 con 0x7f4ae40b1740 2023-03-03T06:25:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f4ada7fc700 1 --2- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:6800/32451,v1:172.21.15.32:6803/32451] conn(0x7f4ac806eb30 0x7f4ac8070f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f4ada7fc700 1 -- 172.21.15.3:0/3877249627 --> [v2:172.21.15.32:6800/32451,v1:172.21.15.32:6803/32451] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ac8071630 con 0x7f4ac806eb30 2023-03-03T06:25:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7f4ada7fc700 1 -- 172.21.15.3:0/3877249627 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ad4053090 con 0x7f4ae40b1740 2023-03-03T06:25:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7f4aeaff9700 1 --2- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:6800/32451,v1:172.21.15.32:6803/32451] conn(0x7f4ac806eb30 0x7f4ac8070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7f4ae97f6700 1 --2- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4ac8064ee0 0x7f4ac8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7fdea2ffd700 1 --2- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fde94064fb0 0x7fde94067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7fdea2ffd700 1 -- 172.21.15.3:0/624479666 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdea4081f10 con 0x7fdeb40aea70 2023-03-03T06:25:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2764007120 con 0x7f2764058b60 2023-03-03T06:25:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.591+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27640571c0 con 0x7f2764058b60 2023-03-03T06:25:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7f2768ff9700 1 -- 172.21.15.3:0/870111000 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c096050 con 0x7f2764058b60 2023-03-03T06:25:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7f2768ff9700 1 -- 172.21.15.3:0/870111000 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c06f740 con 0x7f2764058b60 2023-03-03T06:25:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7fdeb259c700 1 --2- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fde94064fb0 0x7fde94067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7f4aeaff9700 1 --2- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:6800/32451,v1:172.21.15.32:6803/32451] conn(0x7f4ac806eb30 0x7f4ac8070f60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7f4ae97f6700 1 --2- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4ac8064ee0 0x7f4ac8067390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40c4c30 tx=0x7f4ae0007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7fdeb89e6700 1 --2- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:6824/32453,v1:172.21.15.32:6825/32453] conn(0x7fdeb40b1f70 0x7fdeb4067b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7fdeb89e6700 1 -- 172.21.15.3:0/624479666 --> [v2:172.21.15.32:6824/32453,v1:172.21.15.32:6825/32453] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdeb40b4510 con 0x7fdeb40b1f70 2023-03-03T06:25:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7effff59e700 1 -- 172.21.15.3:0/2462062034 --> [v2:172.21.15.3:6824/32269,v1:172.21.15.3:6825/32269] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f00000c1f50 con 0x7effe006ec00 2023-03-03T06:25:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7fdeb259c700 1 --2- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fde94064fb0 0x7fde94067460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00b370 tx=0x7fdeac009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7fdeb2d9d700 1 --2- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:6824/32453,v1:172.21.15.32:6825/32453] conn(0x7fdeb40b1f70 0x7fdeb4067b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7efff6ffd700 1 -- 172.21.15.3:0/2462062034 <== osd.0 v2:172.21.15.3:6824/32269 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f00000c1f50 con 0x7effe006ec00 2023-03-03T06:25:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f274c000fc0 con 0x7f2764058b60 2023-03-03T06:25:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7f2768ff9700 1 -- 172.21.15.3:0/870111000 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f276c06f8a0 con 0x7f2764058b60 2023-03-03T06:25:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7f4ada7fc700 1 -- 172.21.15.3:0/3877249627 <== osd.4 v2:172.21.15.32:6800/32451 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4ac8071630 con 0x7f4ac806eb30 2023-03-03T06:25:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.595+0000 7fdeb2d9d700 1 --2- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:6824/32453,v1:172.21.15.32:6825/32453] conn(0x7fdeb40b1f70 0x7fdeb4067b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7effff59e700 1 -- 172.21.15.3:0/2462062034 >> [v2:172.21.15.3:6824/32269,v1:172.21.15.3:6825/32269] conn(0x7effe006ec00 msgr2=0x7effe0071030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7effff59e700 1 --2- 172.21.15.3:0/2462062034 >> [v2:172.21.15.3:6824/32269,v1:172.21.15.3:6825/32269] conn(0x7effe006ec00 0x7effe0071030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7effff59e700 1 -- 172.21.15.3:0/2462062034 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7effe0064f30 msgr2=0x7effe00673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7effff59e700 1 --2- 172.21.15.3:0/2462062034 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7effe0064f30 0x7effe00673e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f000006dd80 tx=0x7effe8008030 comp rx=0 tx=0).stop 2023-03-03T06:25:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7effff59e700 1 -- 172.21.15.3:0/2462062034 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00000afb60 msgr2=0x7f0000157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7effff59e700 1 --2- 172.21.15.3:0/2462062034 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00000afb60 0x7f0000157bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efff000e810 tx=0x7efff000eb20 comp rx=0 tx=0).stop 2023-03-03T06:25:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7efffed9d700 1 -- 172.21.15.3:0/2462062034 reap_dead start 2023-03-03T06:25:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7effff59e700 1 -- 172.21.15.3:0/2462062034 shutdown_connections 2023-03-03T06:25:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7effff59e700 1 -- 172.21.15.3:0/2462062034 >> 172.21.15.3:0/2462062034 conn(0x7f0000060be0 msgr2=0x7f00000cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7f2768ff9700 1 --2- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2758064fb0 0x7f2758067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7f2768ff9700 1 -- 172.21.15.3:0/870111000 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f276c0fc620 con 0x7f2764058b60 2023-03-03T06:25:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7fdea2ffd700 1 -- 172.21.15.3:0/624479666 <== osd.5 v2:172.21.15.32:6824/32453 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdeb40b4510 con 0x7fdeb40b1f70 2023-03-03T06:25:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7f2768ff9700 1 --2- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:6816/32454,v1:172.21.15.32:6817/32454] conn(0x7f275806ec80 0x7f27580710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7f276bfff700 1 --2- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2758064fb0 0x7f2758067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7f2768ff9700 1 -- 172.21.15.3:0/870111000 --> [v2:172.21.15.32:6816/32454,v1:172.21.15.32:6817/32454] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2758071780 con 0x7f275806ec80 2023-03-03T06:25:45.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7f2768ff9700 1 -- 172.21.15.3:0/870111000 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f276c0fec40 con 0x7f2764058b60 2023-03-03T06:25:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7feb3a596700 1 -- 172.21.15.3:0/2297281263 --> [v2:172.21.15.3:6808/32272,v1:172.21.15.3:6809/32272] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7feb340ca420 con 0x7feb3415d0c0 2023-03-03T06:25:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7fa7a759e700 1 -- 172.21.15.3:0/1344247360 shutdown_connections 2023-03-03T06:25:45.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7fa7a759e700 1 --2- 172.21.15.3:0/1344247360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7a0047c50 0x7fa7a00480a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7fa7a759e700 1 --2- 172.21.15.3:0/1344247360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7a00e6f20 0x7fa7a004f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7fa7a759e700 1 -- 172.21.15.3:0/1344247360 >> 172.21.15.3:0/1344247360 conn(0x7fa7a0051710 msgr2=0x7fa7a0051ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7fa7a759e700 1 -- 172.21.15.3:0/1344247360 shutdown_connections 2023-03-03T06:25:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7fa7a759e700 1 -- 172.21.15.3:0/1344247360 wait complete. 2023-03-03T06:25:45.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7effff59e700 1 -- 172.21.15.3:0/2462062034 shutdown_connections 2023-03-03T06:25:45.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.599+0000 7effff59e700 1 -- 172.21.15.3:0/2462062034 wait complete. 2023-03-03T06:25:45.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7feb30ff9700 1 -- 172.21.15.3:0/2297281263 <== osd.1 v2:172.21.15.3:6808/32272 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7feb340ca420 con 0x7feb3415d0c0 2023-03-03T06:25:45.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7f276bfff700 1 --2- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2758064fb0 0x7f2758067460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2760009750 tx=0x7f2760006cd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7f276b7fe700 1 --2- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:6816/32454,v1:172.21.15.32:6817/32454] conn(0x7f275806ec80 0x7f27580710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7feb1a7fc700 1 -- 172.21.15.3:0/2297281263 >> [v2:172.21.15.3:6808/32272,v1:172.21.15.3:6809/32272] conn(0x7feb3415d0c0 msgr2=0x7feb340c2410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7feb1a7fc700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.3:6808/32272,v1:172.21.15.3:6809/32272] conn(0x7feb3415d0c0 0x7feb340c2410 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7fa7a759e700 1 Processor -- start 2023-03-03T06:25:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7f276b7fe700 1 --2- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:6816/32454,v1:172.21.15.32:6817/32454] conn(0x7f275806ec80 0x7f27580710b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7feb1a7fc700 1 -- 172.21.15.3:0/2297281263 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7feb20064f60 msgr2=0x7feb20067410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7feb1a7fc700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7feb20064f60 0x7feb20067410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009fd0 tx=0x7feb2c00b040 comp rx=0 tx=0).stop 2023-03-03T06:25:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7feb1a7fc700 1 -- 172.21.15.3:0/2297281263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb340aec00 msgr2=0x7feb340c2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7feb1a7fc700 1 --2- 172.21.15.3:0/2297281263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb340aec00 0x7feb340c2fe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feb2400c730 tx=0x7feb2400ca40 comp rx=0 tx=0).stop 2023-03-03T06:25:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7feb38b33700 1 -- 172.21.15.3:0/2297281263 reap_dead start 2023-03-03T06:25:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.607+0000 7feb1a7fc700 1 -- 172.21.15.3:0/2297281263 shutdown_connections 2023-03-03T06:25:45.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.607+0000 7feb1a7fc700 1 -- 172.21.15.3:0/2297281263 >> 172.21.15.3:0/2297281263 conn(0x7feb34060be0 msgr2=0x7feb3406cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7fac24ff9700 1 -- 172.21.15.3:0/3334343514 --> [v2:172.21.15.3:6816/32268,v1:172.21.15.3:6817/32268] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fac10002d60 con 0x7fac1806ec00 2023-03-03T06:25:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.603+0000 7fa7a759e700 1 -- start start 2023-03-03T06:25:45.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.607+0000 7fac26ffd700 1 -- 172.21.15.3:0/3334343514 <== osd.3 v2:172.21.15.3:6816/32268 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fac10002d60 con 0x7fac1806ec00 2023-03-03T06:25:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.607+0000 7f3d4e448700 1 -- 172.21.15.3:0/898193348 --> [v2:172.21.15.3:6800/32271,v1:172.21.15.3:6801/32271] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d48073e90 con 0x7f3d3406eb10 2023-03-03T06:25:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.607+0000 7feb1a7fc700 1 -- 172.21.15.3:0/2297281263 shutdown_connections 2023-03-03T06:25:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.607+0000 7feb1a7fc700 1 -- 172.21.15.3:0/2297281263 wait complete. 2023-03-03T06:25:45.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.607+0000 7f3d44ff9700 1 -- 172.21.15.3:0/898193348 <== osd.2 v2:172.21.15.3:6800/32271 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3d48073e90 con 0x7f3d3406eb10 2023-03-03T06:25:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.607+0000 7fac3cabc700 1 -- 172.21.15.3:0/3334343514 >> [v2:172.21.15.3:6816/32268,v1:172.21.15.3:6817/32268] conn(0x7fac1806ec00 msgr2=0x7fac18071030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.607+0000 7fac3cabc700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.3:6816/32268,v1:172.21.15.3:6817/32268] conn(0x7fac1806ec00 0x7fac18071030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.607+0000 7fa7a759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7a0005fa0 0x7fa7a0044b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.607+0000 7f3d2e7fc700 1 -- 172.21.15.3:0/898193348 >> [v2:172.21.15.3:6800/32271,v1:172.21.15.3:6801/32271] conn(0x7f3d3406eb10 msgr2=0x7f3d34070f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.607+0000 7f3d2e7fc700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.3:6800/32271,v1:172.21.15.3:6801/32271] conn(0x7f3d3406eb10 0x7f3d34070f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.607+0000 7fa7a759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a0047c50 0x7fa7a00491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7a00e6f20 0x7fa7a00e5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fac3cabc700 1 -- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fac18064f30 msgr2=0x7fac180673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7a0005fa0 0x7fa7a0044b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a659c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7a0005fa0 0x7fa7a0044b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:55422/0 (socket says 172.21.15.3:55422) 2023-03-03T06:25:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a0047c50 0x7fa7a00491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a0047c50 0x7fa7a00491d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39786/0 (socket says 172.21.15.3:39786) 2023-03-03T06:25:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7f3d2e7fc700 1 -- 172.21.15.3:0/898193348 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3d34064fb0 msgr2=0x7f3d34067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7f3d2e7fc700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3d34064fb0 0x7f3d34067460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009940 tx=0x7f3d38008030 comp rx=0 tx=0).stop 2023-03-03T06:25:45.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7f3d2e7fc700 1 -- 172.21.15.3:0/898193348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480b0c60 msgr2=0x7f3d480b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7f3d2e7fc700 1 --2- 172.21.15.3:0/898193348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d480b0c60 0x7f3d480b3d50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00e9d0 tx=0x7f3d3c00ece0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fac3cabc700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fac18064f30 0x7fac180673e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fac38069680 tx=0x7fac38072040 comp rx=0 tx=0).stop 2023-03-03T06:25:45.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7f3d4c9e5700 1 -- 172.21.15.3:0/898193348 reap_dead start 2023-03-03T06:25:45.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7f3d2e7fc700 1 -- 172.21.15.3:0/898193348 shutdown_connections 2023-03-03T06:25:45.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7f3d2e7fc700 1 -- 172.21.15.3:0/898193348 >> 172.21.15.3:0/898193348 conn(0x7f3d48060be0 msgr2=0x7f3d48157810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a659c700 1 -- 172.21.15.3:0/3736092487 learned_addr learned my addr 172.21.15.3:0/3736092487 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:45.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a004d400 con 0x7fa7a00e6f20 2023-03-03T06:25:45.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0049a20 con 0x7fa7a0047c50 2023-03-03T06:25:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fac3cabc700 1 -- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac30054a20 msgr2=0x7fac30038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fac3cabc700 1 --2- 172.21.15.3:0/3334343514 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fac30054a20 0x7fac30038e20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac2c015000 tx=0x7fac2c004770 comp rx=0 tx=0).stop 2023-03-03T06:25:45.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fac36d9d700 1 -- 172.21.15.3:0/3334343514 reap_dead start 2023-03-03T06:25:45.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fac3cabc700 1 -- 172.21.15.3:0/3334343514 shutdown_connections 2023-03-03T06:25:45.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fac3cabc700 1 -- 172.21.15.3:0/3334343514 >> 172.21.15.3:0/3334343514 conn(0x7fac30051a20 msgr2=0x7fac300d68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fac3cabc700 1 -- 172.21.15.3:0/3334343514 shutdown_connections 2023-03-03T06:25:45.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fac3cabc700 1 -- 172.21.15.3:0/3334343514 wait complete. 2023-03-03T06:25:45.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7f2768ff9700 1 -- 172.21.15.3:0/870111000 <== osd.7 v2:172.21.15.32:6816/32454 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2758071780 con 0x7f275806ec80 2023-03-03T06:25:45.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7f3d2e7fc700 1 -- 172.21.15.3:0/898193348 shutdown_connections 2023-03-03T06:25:45.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7f3d2e7fc700 1 -- 172.21.15.3:0/898193348 wait complete. 2023-03-03T06:25:45.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a6d9d700 1 --2- 172.21.15.3:0/3736092487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7a00e6f20 0x7fa7a00e5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0049d00 con 0x7fa7a0005fa0 2023-03-03T06:25:45.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a6d9d700 1 -- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7a0005fa0 msgr2=0x7fa7a0044b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a6d9d700 1 --2- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7a0005fa0 0x7fa7a0044b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a6d9d700 1 -- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a0047c50 msgr2=0x7fa7a00491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a6d9d700 1 --2- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7a0047c50 0x7fa7a00491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a6d9d700 1 -- 172.21.15.3:0/3736092487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c0094b0 con 0x7fa7a00e6f20 2023-03-03T06:25:45.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.611+0000 7fa7a6d9d700 1 --2- 172.21.15.3:0/3736092487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7a00e6f20 0x7fa7a00e5df0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8073980 tx=0x7fa7a8072330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.615+0000 7fa796ffd700 1 -- 172.21.15.3:0/3736092487 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8066ec0 con 0x7fa7a00e6f20 2023-03-03T06:25:45.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.615+0000 7fa796ffd700 1 -- 172.21.15.3:0/3736092487 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a8067040 con 0x7fa7a00e6f20 2023-03-03T06:25:45.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.615+0000 7fa796ffd700 1 -- 172.21.15.3:0/3736092487 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8067d50 con 0x7fa7a00e6f20 2023-03-03T06:25:45.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.615+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a00532a0 con 0x7fa7a00e6f20 2023-03-03T06:25:45.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.615+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a0052fc0 con 0x7fa7a00e6f20 2023-03-03T06:25:45.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.615+0000 7fa796ffd700 1 -- 172.21.15.3:0/3736092487 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa7a8067eb0 con 0x7fa7a00e6f20 2023-03-03T06:25:45.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.619+0000 7fa796ffd700 1 --2- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa788064fb0 0x7fa788067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.619+0000 7fa796ffd700 1 -- 172.21.15.3:0/3736092487 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa7a80fb370 con 0x7fa7a00e6f20 2023-03-03T06:25:45.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.619+0000 7fa7a659c700 1 --2- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa788064fb0 0x7fa788067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.623+0000 7fa7a659c700 1 --2- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa788064fb0 0x7fa788067460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa798009ae0 tx=0x7fa798009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.623+0000 7fa7a759e700 1 --2- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:6801/32452,v1:172.21.15.32:6802/32452] conn(0x7fa7a00345d0 0x7fa7a004ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:45.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.623+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 --> [v2:172.21.15.32:6801/32452,v1:172.21.15.32:6802/32452] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7a00485e0 con 0x7fa7a00345d0 2023-03-03T06:25:45.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.627+0000 7fa7a5d9b700 1 --2- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:6801/32452,v1:172.21.15.32:6802/32452] conn(0x7fa7a00345d0 0x7fa7a004ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:45.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.627+0000 7f4aec25b700 1 -- 172.21.15.3:0/3877249627 --> [v2:172.21.15.32:6800/32451,v1:172.21.15.32:6803/32451] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4acc002d60 con 0x7f4ac806eb30 2023-03-03T06:25:45.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.627+0000 7fa7a5d9b700 1 --2- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:6801/32452,v1:172.21.15.32:6802/32452] conn(0x7fa7a00345d0 0x7fa7a004ac00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:45.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.627+0000 7f4ada7fc700 1 -- 172.21.15.3:0/3877249627 <== osd.4 v2:172.21.15.32:6800/32451 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4acc002d60 con 0x7f4ac806eb30 2023-03-03T06:25:45.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.627+0000 7f4aec25b700 1 -- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:6800/32451,v1:172.21.15.32:6803/32451] conn(0x7f4ac806eb30 msgr2=0x7f4ac8070f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.627+0000 7f4aec25b700 1 --2- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:6800/32451,v1:172.21.15.32:6803/32451] conn(0x7f4ac806eb30 0x7f4ac8070f60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.627+0000 7fa796ffd700 1 -- 172.21.15.3:0/3736092487 <== osd.6 v2:172.21.15.32:6801/32452 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa7a00485e0 con 0x7fa7a00345d0 2023-03-03T06:25:45.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.631+0000 7f4aec25b700 1 -- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4ac8064ee0 msgr2=0x7f4ac8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.631+0000 7f4aec25b700 1 --2- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4ac8064ee0 0x7f4ac8067390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40c4c30 tx=0x7f4ae0007ec0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.631+0000 7f4aec25b700 1 -- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ae40b1740 msgr2=0x7f4ae40b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.631+0000 7f4aec25b700 1 --2- 172.21.15.3:0/3877249627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ae40b1740 0x7f4ae40b0130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400bed0 tx=0x7f4ad400bf00 comp rx=0 tx=0).stop 2023-03-03T06:25:45.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.631+0000 7f4aeaff9700 1 -- 172.21.15.3:0/3877249627 reap_dead start 2023-03-03T06:25:45.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.631+0000 7f4aec25b700 1 -- 172.21.15.3:0/3877249627 shutdown_connections 2023-03-03T06:25:45.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.631+0000 7f4aec25b700 1 -- 172.21.15.3:0/3877249627 >> 172.21.15.3:0/3877249627 conn(0x7f4ae406a110 msgr2=0x7f4ae4071260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.635+0000 7f4aec25b700 1 -- 172.21.15.3:0/3877249627 shutdown_connections 2023-03-03T06:25:45.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.635+0000 7f4aec25b700 1 -- 172.21.15.3:0/3877249627 wait complete. 2023-03-03T06:25:45.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.635+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 --> [v2:172.21.15.32:6816/32454,v1:172.21.15.32:6817/32454] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f274c002d60 con 0x7f275806ec80 2023-03-03T06:25:45.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.635+0000 7f2768ff9700 1 -- 172.21.15.3:0/870111000 <== osd.7 v2:172.21.15.32:6816/32454 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f274c002d60 con 0x7f275806ec80 2023-03-03T06:25:45.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.635+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:6816/32454,v1:172.21.15.32:6817/32454] conn(0x7f275806ec80 msgr2=0x7f27580710b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.635+0000 7f2772550700 1 --2- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:6816/32454,v1:172.21.15.32:6817/32454] conn(0x7f275806ec80 0x7f27580710b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.635+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2758064fb0 msgr2=0x7f2758067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.635+0000 7f2772550700 1 --2- 172.21.15.3:0/870111000 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2758064fb0 0x7f2758067460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2760009750 tx=0x7f2760006cd0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.635+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2764058b60 msgr2=0x7f276400a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.635+0000 7f2772550700 1 --2- 172.21.15.3:0/870111000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2764058b60 0x7f276400a390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f276c073980 tx=0x7f276c069c30 comp rx=0 tx=0).stop 2023-03-03T06:25:45.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.635+0000 7f2770aed700 1 -- 172.21.15.3:0/870111000 reap_dead start 2023-03-03T06:25:45.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.635+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 shutdown_connections 2023-03-03T06:25:45.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.635+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 >> 172.21.15.3:0/870111000 conn(0x7f276400f300 msgr2=0x7f2764059780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.639+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 shutdown_connections 2023-03-03T06:25:45.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.639+0000 7f2772550700 1 -- 172.21.15.3:0/870111000 wait complete. 2023-03-03T06:25:45.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.643+0000 7fdeb89e6700 1 -- 172.21.15.3:0/624479666 --> [v2:172.21.15.32:6824/32453,v1:172.21.15.32:6825/32453] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdeb4073e90 con 0x7fdeb40b1f70 2023-03-03T06:25:45.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.643+0000 7fdea2ffd700 1 -- 172.21.15.3:0/624479666 <== osd.5 v2:172.21.15.32:6824/32453 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdeb4073e90 con 0x7fdeb40b1f70 2023-03-03T06:25:45.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.643+0000 7fdeb89e6700 1 -- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:6824/32453,v1:172.21.15.32:6825/32453] conn(0x7fdeb40b1f70 msgr2=0x7fdeb4067b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.643+0000 7fdeb89e6700 1 --2- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:6824/32453,v1:172.21.15.32:6825/32453] conn(0x7fdeb40b1f70 0x7fdeb4067b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.643+0000 7fdeb89e6700 1 -- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fde94064fb0 msgr2=0x7fde94067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.643+0000 7fdeb89e6700 1 --2- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fde94064fb0 0x7fde94067460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00b370 tx=0x7fdeac009000 comp rx=0 tx=0).stop 2023-03-03T06:25:45.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.647+0000 7fdeb89e6700 1 -- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb40aea70 msgr2=0x7fdeb40c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.647+0000 7fdeb89e6700 1 --2- 172.21.15.3:0/624479666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdeb40aea70 0x7fdeb40c7a30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009fd0 tx=0x7fdea400b6c0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.698 INFO:teuthology.orchestra.run.smithi003.stdout:55834574860 2023-03-03T06:25:45.698 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-03-03T06:25:45.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.647+0000 7fdeb2d9d700 1 -- 172.21.15.3:0/624479666 reap_dead start 2023-03-03T06:25:45.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.647+0000 7fdeb89e6700 1 -- 172.21.15.3:0/624479666 shutdown_connections 2023-03-03T06:25:45.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.647+0000 7fdeb89e6700 1 -- 172.21.15.3:0/624479666 >> 172.21.15.3:0/624479666 conn(0x7fdeb4070a30 msgr2=0x7fdeb40af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.647+0000 7fdeb89e6700 1 -- 172.21.15.3:0/624479666 shutdown_connections 2023-03-03T06:25:45.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.647+0000 7fdeb89e6700 1 -- 172.21.15.3:0/624479666 wait complete. 2023-03-03T06:25:45.700 INFO:teuthology.orchestra.run.smithi003.stdout:55834574861 2023-03-03T06:25:45.701 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-03-03T06:25:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.655+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 --> [v2:172.21.15.32:6801/32452,v1:172.21.15.32:6802/32452] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa7a0005460 con 0x7fa7a00345d0 2023-03-03T06:25:45.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.655+0000 7fa796ffd700 1 -- 172.21.15.3:0/3736092487 <== osd.6 v2:172.21.15.32:6801/32452 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa7a0005460 con 0x7fa7a00345d0 2023-03-03T06:25:45.702 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2023-03-03T06:25:45.702 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-03-03T06:25:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.655+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:6801/32452,v1:172.21.15.32:6802/32452] conn(0x7fa7a00345d0 msgr2=0x7fa7a004ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.655+0000 7fa7a759e700 1 --2- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:6801/32452,v1:172.21.15.32:6802/32452] conn(0x7fa7a00345d0 0x7fa7a004ac00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:45.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.655+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa788064fb0 msgr2=0x7fa788067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.655+0000 7fa7a759e700 1 --2- 172.21.15.3:0/3736092487 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa788064fb0 0x7fa788067460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa798009ae0 tx=0x7fa798009240 comp rx=0 tx=0).stop 2023-03-03T06:25:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.655+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7a00e6f20 msgr2=0x7fa7a00e5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.655+0000 7fa7a759e700 1 --2- 172.21.15.3:0/3736092487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7a00e6f20 0x7fa7a00e5df0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8073980 tx=0x7fa7a8072330 comp rx=0 tx=0).stop 2023-03-03T06:25:45.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.659+0000 7fa7a6d9d700 1 -- 172.21.15.3:0/3736092487 reap_dead start 2023-03-03T06:25:45.704 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2023-03-03T06:25:45.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.659+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 shutdown_connections 2023-03-03T06:25:45.705 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-03-03T06:25:45.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.659+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 >> 172.21.15.3:0/3736092487 conn(0x7fa7a0051710 msgr2=0x7fa7a0037ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:45.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.659+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 shutdown_connections 2023-03-03T06:25:45.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:45.659+0000 7fa7a759e700 1 -- 172.21.15.3:0/3736092487 wait complete. 2023-03-03T06:25:45.706 INFO:teuthology.orchestra.run.smithi003.stdout:55834574860 2023-03-03T06:25:45.707 INFO:teuthology.orchestra.run.smithi003.stdout:55834574860 2023-03-03T06:25:45.708 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-03-03T06:25:45.708 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-03-03T06:25:45.709 INFO:teuthology.orchestra.run.smithi003.stdout:55834574860 2023-03-03T06:25:45.709 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-03-03T06:25:45.710 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2023-03-03T06:25:45.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-03-03T06:25:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.023+0000 7f60867c6700 1 Processor -- start 2023-03-03T06:25:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.023+0000 7f68092d8700 1 Processor -- start 2023-03-03T06:25:46.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.023+0000 7f60867c6700 1 -- start start 2023-03-03T06:25:46.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.023+0000 7f60867c6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60780e2970 0x7f6078006b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.023+0000 7f68092d8700 1 -- start start 2023-03-03T06:25:46.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f68092d8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f680406e5c0 0x7f680406e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f68092d8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6804160d20 con 0x7f6804070a00 2023-03-03T06:25:46.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f68092d8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680406eed0 con 0x7f680415c8c0 2023-03-03T06:25:46.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f68092d8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68040cf3e0 con 0x7f680406e5c0 2023-03-03T06:25:46.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f60867c6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f607803b0f0 0x7f6078057720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f607ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60780e2970 0x7f6078006b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f607f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6078053520 0x7f607803a9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33672/0 (socket says 172.21.15.3:33672) 2023-03-03T06:25:46.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f607ffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f60780e2970 0x7f6078006b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39796/0 (socket says 172.21.15.3:39796) 2023-03-03T06:25:46.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f607ffff700 1 -- 172.21.15.3:0/3448106060 learned_addr learned my addr 172.21.15.3:0/3448106060 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:46.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6084d63700 1 --2- 172.21.15.3:0/3448106060 >> v2:172.21.15.3:3300/0 conn(0x7f607803b0f0 0x7f6078057720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f60867c6700 1 -- 172.21.15.3:0/3448106060 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6078053a70 con 0x7f6078053520 2023-03-03T06:25:46.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f60867c6700 1 -- 172.21.15.3:0/3448106060 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6078053d50 con 0x7f607803b0f0 2023-03-03T06:25:46.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f60867c6700 1 -- 172.21.15.3:0/3448106060 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6078056c40 con 0x7f60780e2970 2023-03-03T06:25:46.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6803fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6804070a00 0x7f6804070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33688/0 (socket says 172.21.15.3:33688) 2023-03-03T06:25:46.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6803fff700 1 -- 172.21.15.3:0/2316656574 learned_addr learned my addr 172.21.15.3:0/2316656574 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:46.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6084d63700 1 -- 172.21.15.3:0/3448106060 >> v2:172.21.15.32:3300/0 conn(0x7f60780e2970 msgr2=0x7f6078006b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:46.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6084d63700 1 --2- 172.21.15.3:0/3448106060 >> v2:172.21.15.32:3300/0 conn(0x7f60780e2970 0x7f6078006b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6084d63700 1 -- 172.21.15.3:0/3448106060 >> v1:172.21.15.3:6789/0 conn(0x7f6078053520 legacy=0x7f607803a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6084d63700 1 -- 172.21.15.3:0/3448106060 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6078056f20 con 0x7f607803b0f0 2023-03-03T06:25:46.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6802ffd700 1 --2- 172.21.15.3:0/2316656574 >> v2:172.21.15.32:3300/0 conn(0x7f680406e5c0 0x7f680406e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6084d63700 1 --2- 172.21.15.3:0/3448106060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f607803b0f0 0x7f6078057720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f608004f390 tx=0x7f60800617d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfb46bc55d63d8d server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6801ffb700 1 -- 172.21.15.3:0/2316656574 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875338547 0 0) 0x7f6804160d20 con 0x7f6804070a00 2023-03-03T06:25:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6801ffb700 1 -- 172.21.15.3:0/2316656574 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ec0035d0 con 0x7f6804070a00 2023-03-03T06:25:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6801ffb700 1 -- 172.21.15.3:0/2316656574 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778733198 0 0) 0x7f680406eed0 con 0x7f680415c8c0 2023-03-03T06:25:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6801ffb700 1 -- 172.21.15.3:0/2316656574 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6804160d20 con 0x7f680415c8c0 2023-03-03T06:25:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f607effd700 1 -- 172.21.15.3:0/3448106060 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080066cc0 con 0x7f607803b0f0 2023-03-03T06:25:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f6802ffd700 1 -- 172.21.15.3:0/2316656574 >> v1:172.21.15.32:6789/0 conn(0x7f680415c8c0 legacy=0x7f68040cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f6801ffb700 1 -- 172.21.15.3:0/2316656574 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1476418418 0 0) 0x7f67ec0035d0 con 0x7f6804070a00 2023-03-03T06:25:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f6802ffd700 1 -- 172.21.15.3:0/2316656574 >> v1:172.21.15.3:6789/0 conn(0x7f6804070a00 legacy=0x7f6804070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f6802ffd700 1 -- 172.21.15.3:0/2316656574 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680415fe30 con 0x7f680406e5c0 2023-03-03T06:25:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f607effd700 1 -- 172.21.15.3:0/3448106060 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6080066e40 con 0x7f607803b0f0 2023-03-03T06:25:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.027+0000 7f607effd700 1 -- 172.21.15.3:0/3448106060 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60800720c0 con 0x7f607803b0f0 2023-03-03T06:25:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f05e686d700 1 Processor -- start 2023-03-03T06:25:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f60867c6700 1 -- 172.21.15.3:0/3448106060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f607803b0f0 msgr2=0x7f6078057720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f60867c6700 1 --2- 172.21.15.3:0/3448106060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f607803b0f0 0x7f6078057720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f608004f390 tx=0x7f60800617d0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f6802ffd700 1 --2- 172.21.15.3:0/2316656574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680406e5c0 0x7f680406e990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0025e0 tx=0x7f67fc00c190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c597cba4cb196638 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f60867c6700 1 -- 172.21.15.3:0/3448106060 shutdown_connections 2023-03-03T06:25:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f60867c6700 1 --2- 172.21.15.3:0/3448106060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f607803b0f0 0x7f6078057720 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f60867c6700 1 --2- 172.21.15.3:0/3448106060 >> v2:172.21.15.32:3300/0 conn(0x7f60780e2970 0x7f6078006b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f60867c6700 1 -- 172.21.15.3:0/3448106060 >> 172.21.15.3:0/3448106060 conn(0x7f60780559e0 msgr2=0x7f6078055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f6801ffb700 1 -- 172.21.15.3:0/2316656574 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4001670 con 0x7f680406e5c0 2023-03-03T06:25:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f6801ffb700 1 -- 172.21.15.3:0/2316656574 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc00c940 con 0x7f680406e5c0 2023-03-03T06:25:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f6801ffb700 1 -- 172.21.15.3:0/2316656574 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc00ccd0 con 0x7f680406e5c0 2023-03-03T06:25:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f60867c6700 1 -- 172.21.15.3:0/3448106060 shutdown_connections 2023-03-03T06:25:46.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f60867c6700 1 -- 172.21.15.3:0/3448106060 wait complete. 2023-03-03T06:25:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f68092d8700 1 -- 172.21.15.3:0/2316656574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680406e5c0 msgr2=0x7f680406e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f68092d8700 1 --2- 172.21.15.3:0/2316656574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680406e5c0 0x7f680406e990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0025e0 tx=0x7f67fc00c190 comp rx=0 tx=0).stop 2023-03-03T06:25:46.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f953bbba700 1 Processor -- start 2023-03-03T06:25:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f68092d8700 1 -- 172.21.15.3:0/2316656574 shutdown_connections 2023-03-03T06:25:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f68092d8700 1 --2- 172.21.15.3:0/2316656574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f680406e5c0 0x7f680406e990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f68092d8700 1 -- 172.21.15.3:0/2316656574 >> 172.21.15.3:0/2316656574 conn(0x7f68040d0310 msgr2=0x7f68040d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f68092d8700 1 -- 172.21.15.3:0/2316656574 shutdown_connections 2023-03-03T06:25:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f05e686d700 1 -- start start 2023-03-03T06:25:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f05e686d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05d80e6f20 0x7f05d804f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f05e686d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d8047c50 0x7f05d80480a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f05e686d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f05d8005fa0 0x7f05d8006420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f68092d8700 1 -- 172.21.15.3:0/2316656574 wait complete. 2023-03-03T06:25:46.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e586b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05d80e6f20 0x7f05d804f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e586b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05d80e6f20 0x7f05d804f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39806/0 (socket says 172.21.15.3:39806) 2023-03-03T06:25:46.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e606c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f05d8005fa0 0x7f05d8006420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e606c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f05d8005fa0 0x7f05d8006420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43450/0 (socket says 172.21.15.3:43450) 2023-03-03T06:25:46.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e586b700 1 -- 172.21.15.3:0/3207737533 learned_addr learned my addr 172.21.15.3:0/3207737533 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:46.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e686d700 1 -- 172.21.15.3:0/3207737533 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d80485e0 con 0x7f05d8005fa0 2023-03-03T06:25:46.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f60867c6700 1 Processor -- start 2023-03-03T06:25:46.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e686d700 1 -- 172.21.15.3:0/3207737533 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d8053610 con 0x7f05d80e6f20 2023-03-03T06:25:46.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e686d700 1 -- 172.21.15.3:0/3207737533 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d8052a00 con 0x7f05d8047c50 2023-03-03T06:25:46.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f60867c6700 1 -- start start 2023-03-03T06:25:46.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f42e80c8700 1 Processor -- start 2023-03-03T06:25:46.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e606c700 1 -- 172.21.15.3:0/3207737533 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d8047c50 msgr2=0x7f05d80480a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-03T06:25:46.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e606c700 1 --2- 172.21.15.3:0/3207737533 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d8047c50 0x7f05d80480a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e606c700 1 -- 172.21.15.3:0/3207737533 >> v2:172.21.15.32:3300/0 conn(0x7f05d80e6f20 msgr2=0x7f05d804f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:46.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e606c700 1 --2- 172.21.15.3:0/3207737533 >> v2:172.21.15.32:3300/0 conn(0x7f05d80e6f20 0x7f05d804f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e606c700 1 -- 172.21.15.3:0/3207737533 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d8052ce0 con 0x7f05d8005fa0 2023-03-03T06:25:46.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f60867c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f607803b0f0 0x7f60780ddea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f60867c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6078053520 0x7f60780dc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f60867c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60780e2970 0x7f60780e08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f60867c6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780dce00 con 0x7f6078053520 2023-03-03T06:25:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f60867c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780dd0e0 con 0x7f60780e2970 2023-03-03T06:25:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f60867c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60780dd3c0 con 0x7f607803b0f0 2023-03-03T06:25:46.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7fa202d4a700 1 Processor -- start 2023-03-03T06:25:46.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f42e80c8700 1 -- start start 2023-03-03T06:25:46.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f05e606c700 1 --2- 172.21.15.3:0/3207737533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05d8005fa0 0x7f05d8006420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f05d800be30 tx=0x7f05dc009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89ddace849ba557 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f68092d8700 1 Processor -- start 2023-03-03T06:25:46.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.031+0000 7f953bbba700 1 -- start start 2023-03-03T06:25:46.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f953bbba700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9534160880 0x7f95340c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f953bbba700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95340adec0 0x7f95340ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f953bbba700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95340ce190 con 0x7f95340aea70 2023-03-03T06:25:46.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f953bbba700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95340c99d0 con 0x7f95340adec0 2023-03-03T06:25:46.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f953bbba700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95340cd580 con 0x7f9534160880 2023-03-03T06:25:46.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f607ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f607803b0f0 0x7f60780ddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f607ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f607803b0f0 0x7f60780ddea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:55430/0 (socket says 172.21.15.3:55430) 2023-03-03T06:25:46.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f607ffff700 1 -- 172.21.15.3:0/1208361213 learned_addr learned my addr 172.21.15.3:0/1208361213 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:46.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f607f7fe700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6078053520 0x7f60780dc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7f68092d8700 1 -- start start 2023-03-03T06:25:46.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.035+0000 7fa202d4a700 1 -- start start 2023-03-03T06:25:46.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f9539956700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9534160880 0x7f95340c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f953a157700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95340adec0 0x7f95340ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f953a157700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f95340adec0 0x7f95340ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43460/0 (socket says 172.21.15.3:43460) 2023-03-03T06:25:46.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f953a157700 1 -- 172.21.15.3:0/2408412954 learned_addr learned my addr 172.21.15.3:0/2408412954 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:46.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f6084d63700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60780e2970 0x7f60780e08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f607ffff700 1 -- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60780e2970 msgr2=0x7f60780e08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f607ffff700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60780e2970 0x7f60780e08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f607ffff700 1 -- 172.21.15.3:0/1208361213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6078053520 msgr2=0x7f60780dc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f607ffff700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6078053520 0x7f60780dc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f607ffff700 1 -- 172.21.15.3:0/1208361213 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6080066950 con 0x7f607803b0f0 2023-03-03T06:25:46.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f05e4869700 1 -- 172.21.15.3:0/3207737533 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc011830 con 0x7f05d8005fa0 2023-03-03T06:25:46.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f05e4869700 1 -- 172.21.15.3:0/3207737533 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc011990 con 0x7f05d8005fa0 2023-03-03T06:25:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f05e4869700 1 -- 172.21.15.3:0/3207737533 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc011d20 con 0x7f05d8005fa0 2023-03-03T06:25:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f9539956700 1 -- 172.21.15.3:0/2408412954 >> v2:172.21.15.3:3300/0 conn(0x7f95340adec0 msgr2=0x7f95340ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f9539956700 1 --2- 172.21.15.3:0/2408412954 >> v2:172.21.15.3:3300/0 conn(0x7f95340adec0 0x7f95340ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f9539956700 1 -- 172.21.15.3:0/2408412954 >> v1:172.21.15.3:6789/0 conn(0x7f95340aea70 legacy=0x7f95340ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f9539956700 1 -- 172.21.15.3:0/2408412954 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95340cd860 con 0x7f9534160880 2023-03-03T06:25:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68092d8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f680406e5c0 0x7f68040b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68092d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6804070a00 0x7f68040c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68092d8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f680415c8c0 0x7f680414aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68092d8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680415bf00 con 0x7f680406e5c0 2023-03-03T06:25:46.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68092d8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68040b6010 con 0x7f6804070a00 2023-03-03T06:25:46.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68092d8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68040b62f0 con 0x7f680415c8c0 2023-03-03T06:25:46.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68027fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6804070a00 0x7f68040c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68027fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6804070a00 0x7f68040c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39836/0 (socket says 172.21.15.3:39836) 2023-03-03T06:25:46.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68027fc700 1 -- 172.21.15.3:0/1028386761 learned_addr learned my addr 172.21.15.3:0/1028386761 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:46.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f6803fff700 1 --2- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f680415c8c0 0x7f680414aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68027fc700 1 -- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f680415c8c0 msgr2=0x7f680414aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68027fc700 1 --2- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f680415c8c0 0x7f680414aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68027fc700 1 -- 172.21.15.3:0/1028386761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f680406e5c0 msgr2=0x7f68040b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68027fc700 1 --2- 172.21.15.3:0/1028386761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f680406e5c0 0x7f68040b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f68027fc700 1 -- 172.21.15.3:0/1028386761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc007920 con 0x7f6804070a00 2023-03-03T06:25:46.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f68027fc700 1 --2- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6804070a00 0x7f68040c9490 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f67f800c490 tx=0x7f67f800c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f67eb7fe700 1 -- 172.21.15.3:0/1028386761 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f80052d0 con 0x7f6804070a00 2023-03-03T06:25:46.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f67eb7fe700 1 -- 172.21.15.3:0/1028386761 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f8005430 con 0x7f6804070a00 2023-03-03T06:25:46.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f67eb7fe700 1 -- 172.21.15.3:0/1028386761 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f80055a0 con 0x7f6804070a00 2023-03-03T06:25:46.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f68092d8700 1 -- 172.21.15.3:0/1028386761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68041603f0 con 0x7f6804070a00 2023-03-03T06:25:46.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f68092d8700 1 -- 172.21.15.3:0/1028386761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6804160110 con 0x7f6804070a00 2023-03-03T06:25:46.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f05e686d700 1 -- 172.21.15.3:0/3207737533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05d8005fa0 msgr2=0x7f05d8006420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f05e686d700 1 --2- 172.21.15.3:0/3207737533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05d8005fa0 0x7f05d8006420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f05d800be30 tx=0x7f05dc009b30 comp rx=0 tx=0).stop 2023-03-03T06:25:46.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f05e686d700 1 -- 172.21.15.3:0/3207737533 shutdown_connections 2023-03-03T06:25:46.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f05e686d700 1 --2- 172.21.15.3:0/3207737533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05d8005fa0 0x7f05d8006420 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f05e686d700 1 --2- 172.21.15.3:0/3207737533 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d8047c50 0x7f05d80480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f05e686d700 1 --2- 172.21.15.3:0/3207737533 >> v2:172.21.15.32:3300/0 conn(0x7f05d80e6f20 0x7f05d804f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f05e686d700 1 -- 172.21.15.3:0/3207737533 >> 172.21.15.3:0/3207737533 conn(0x7f05d8051710 msgr2=0x7f05d8051ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f05e686d700 1 -- 172.21.15.3:0/3207737533 shutdown_connections 2023-03-03T06:25:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f05e686d700 1 -- 172.21.15.3:0/3207737533 wait complete. 2023-03-03T06:25:46.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f05e686d700 1 Processor -- start 2023-03-03T06:25:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f9539956700 1 --2- 172.21.15.3:0/2408412954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9534160880 0x7f95340c9dc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f953406a940 tx=0x7f952c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa32467e6c9dd7d6 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f9538954700 1 -- 172.21.15.3:0/2408412954 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c00e070 con 0x7f9534160880 2023-03-03T06:25:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f9538954700 1 -- 172.21.15.3:0/2408412954 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952c00bae0 con 0x7f9534160880 2023-03-03T06:25:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f9538954700 1 -- 172.21.15.3:0/2408412954 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c00be70 con 0x7f9534160880 2023-03-03T06:25:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f953bbba700 1 -- 172.21.15.3:0/2408412954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9534160880 msgr2=0x7f95340c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f953bbba700 1 --2- 172.21.15.3:0/2408412954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9534160880 0x7f95340c9dc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f953406a940 tx=0x7f952c00b330 comp rx=0 tx=0).stop 2023-03-03T06:25:46.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f953bbba700 1 -- 172.21.15.3:0/2408412954 shutdown_connections 2023-03-03T06:25:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f953bbba700 1 --2- 172.21.15.3:0/2408412954 >> v2:172.21.15.3:3300/0 conn(0x7f95340adec0 0x7f95340ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f953bbba700 1 --2- 172.21.15.3:0/2408412954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9534160880 0x7f95340c9dc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f953bbba700 1 -- 172.21.15.3:0/2408412954 >> 172.21.15.3:0/2408412954 conn(0x7f9534070a30 msgr2=0x7f9534070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f607ffff700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f607803b0f0 0x7f60780ddea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f607400e890 tx=0x7f607400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f607cff9700 1 -- 172.21.15.3:0/1208361213 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074009620 con 0x7f607803b0f0 2023-03-03T06:25:46.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f607cff9700 1 -- 172.21.15.3:0/1208361213 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607400ee90 con 0x7f607803b0f0 2023-03-03T06:25:46.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f607cff9700 1 -- 172.21.15.3:0/1208361213 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607400ca90 con 0x7f607803b0f0 2023-03-03T06:25:46.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f60867c6700 1 -- 172.21.15.3:0/1208361213 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60780070b0 con 0x7f607803b0f0 2023-03-03T06:25:46.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f60867c6700 1 -- 172.21.15.3:0/1208361213 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6078057200 con 0x7f607803b0f0 2023-03-03T06:25:46.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f60867c6700 1 -- 172.21.15.3:0/1208361213 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6078057c60 con 0x7f607803b0f0 2023-03-03T06:25:46.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7fa202d4a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa1fc0cf230 0x7fa1fc158310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7fa202d4a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1fc0cce70 con 0x7fa1fc06d390 2023-03-03T06:25:46.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7fa202d4a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1fc0cd3b0 con 0x7fa1fc066cc0 2023-03-03T06:25:46.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7fa202d4a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc158850 con 0x7fa1fc0cf230 2023-03-03T06:25:46.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7fa1fbfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa1fc06d390 0x7fa1fc0ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33716/0 (socket says 172.21.15.3:33716) 2023-03-03T06:25:46.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7fa1fbfff700 1 -- 172.21.15.3:0/4053674556 learned_addr learned my addr 172.21.15.3:0/4053674556 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:46.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7fa2012e7700 1 --2- 172.21.15.3:0/4053674556 >> v2:172.21.15.3:3300/0 conn(0x7fa1fc0cf230 0x7fa1fc158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7fa2012e7700 1 -- 172.21.15.3:0/4053674556 >> v1:172.21.15.32:6789/0 conn(0x7fa1fc066cc0 legacy=0x7fa1fc0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:46.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7fa2012e7700 1 -- 172.21.15.3:0/4053674556 >> v1:172.21.15.3:6789/0 conn(0x7fa1fc06d390 legacy=0x7fa1fc0ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7fa2012e7700 1 -- 172.21.15.3:0/4053674556 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc15f2c0 con 0x7fa1fc0cf230 2023-03-03T06:25:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7faafbae6700 1 Processor -- start 2023-03-03T06:25:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7faafbae6700 1 -- start start 2023-03-03T06:25:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7faafbae6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faaf406ecd0 0x7faaf40c9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7faafbae6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf4164070 con 0x7faaf40c9e90 2023-03-03T06:25:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7faafbae6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf4164310 con 0x7faaf4163ca0 2023-03-03T06:25:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7faafbae6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf4164f90 con 0x7faaf406ecd0 2023-03-03T06:25:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7faaf9882700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faaf4163ca0 0x7faaf406e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:56560/0 (socket says 172.21.15.3:56560) 2023-03-03T06:25:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7faaf9882700 1 -- 172.21.15.3:0/3255647801 learned_addr learned my addr 172.21.15.3:0/3255647801 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7faaf9081700 1 --2- 172.21.15.3:0/3255647801 >> v2:172.21.15.32:3300/0 conn(0x7faaf406ecd0 0x7faaf40c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.039+0000 7f42e80c8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42d800c310 0x7f42d80367b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f42e80c8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42d8037ff0 0x7f42d804f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f42e80c8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d8036cf0 con 0x7f42d8037b80 2023-03-03T06:25:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f42e80c8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d8036f20 con 0x7f42d800c310 2023-03-03T06:25:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f42e80c8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d8052cb0 con 0x7f42d8037ff0 2023-03-03T06:25:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f42e6e66700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42d8037ff0 0x7f42d804f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f42e6e66700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42d8037ff0 0x7f42d804f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39840/0 (socket says 172.21.15.3:39840) 2023-03-03T06:25:46.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f42e6e66700 1 -- 172.21.15.3:0/3848422445 learned_addr learned my addr 172.21.15.3:0/3848422445 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:46.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f42e6e66700 1 -- 172.21.15.3:0/3848422445 >> v2:172.21.15.3:3300/0 conn(0x7f42d800c310 msgr2=0x7f42d80367b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:46.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f42e6e66700 1 --2- 172.21.15.3:0/3848422445 >> v2:172.21.15.3:3300/0 conn(0x7f42d800c310 0x7f42d80367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f42e6e66700 1 -- 172.21.15.3:0/3848422445 >> v1:172.21.15.3:6789/0 conn(0x7f42d8037b80 legacy=0x7f42d8053790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:46.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f42e6e66700 1 -- 172.21.15.3:0/3848422445 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d8052f90 con 0x7f42d8037ff0 2023-03-03T06:25:46.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f68092d8700 1 -- 172.21.15.3:0/1028386761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68041606d0 con 0x7f6804070a00 2023-03-03T06:25:46.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f42e6e66700 1 --2- 172.21.15.3:0/3848422445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d8037ff0 0x7f42d804f380 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f42d804bd90 tx=0x7f42d0009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdde395439956eb server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7f05e686d700 1 -- start start 2023-03-03T06:25:46.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f953bbba700 1 -- 172.21.15.3:0/2408412954 shutdown_connections 2023-03-03T06:25:46.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f953bbba700 1 -- 172.21.15.3:0/2408412954 wait complete. 2023-03-03T06:25:46.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f607cff9700 1 -- 172.21.15.3:0/1208361213 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6074019030 con 0x7f607803b0f0 2023-03-03T06:25:46.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.043+0000 7fa2012e7700 1 --2- 172.21.15.3:0/4053674556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1fc0cf230 0x7fa1fc158310 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009dc0 tx=0x7fa1f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a834e03fb0a0e3 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7fa1fb7fe700 1 -- 172.21.15.3:0/4053674556 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f400e070 con 0x7fa1fc0cf230 2023-03-03T06:25:46.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7fa1fb7fe700 1 -- 172.21.15.3:0/4053674556 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f400bb50 con 0x7fa1fc0cf230 2023-03-03T06:25:46.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7fa1fb7fe700 1 -- 172.21.15.3:0/4053674556 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4003a40 con 0x7fa1fc0cf230 2023-03-03T06:25:46.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7faaf9081700 1 -- 172.21.15.3:0/3255647801 >> v1:172.21.15.32:6789/0 conn(0x7faaf4163ca0 legacy=0x7faaf406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7faaf9081700 1 -- 172.21.15.3:0/3255647801 >> v1:172.21.15.3:6789/0 conn(0x7faaf40c9e90 legacy=0x7faaf406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7faaf8880700 1 -- 172.21.15.3:0/3255647801 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161548399 0 0) 0x7faaf4164310 con 0x7faaf4163ca0 2023-03-03T06:25:46.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7faaf9081700 1 -- 172.21.15.3:0/3255647801 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf41645f0 con 0x7faaf406ecd0 2023-03-03T06:25:46.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f42e4e62700 1 -- 172.21.15.3:0/3848422445 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0004230 con 0x7f42d8037ff0 2023-03-03T06:25:46.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7faaf9081700 1 --2- 172.21.15.3:0/3255647801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf406ecd0 0x7faaf40c9950 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faaec0025e0 tx=0x7faaec00c190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3657fd4dbc0edd97 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e686d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05d8005fa0 0x7f05d80e61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e586b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05d8005fa0 0x7f05d80e61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e586b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05d8005fa0 0x7f05d80e61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43526/0 (socket says 172.21.15.3:43526) 2023-03-03T06:25:46.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e586b700 1 -- 172.21.15.3:0/1253084214 learned_addr learned my addr 172.21.15.3:0/1253084214 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:46.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e686d700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d8047c50 0x7f05d8185120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e686d700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05d80e6f20 0x7f05d8039cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e686d700 1 -- 172.21.15.3:0/1253084214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d804af70 con 0x7f05d8005fa0 2023-03-03T06:25:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e686d700 1 -- 172.21.15.3:0/1253084214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d804a810 con 0x7f05d80e6f20 2023-03-03T06:25:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e686d700 1 -- 172.21.15.3:0/1253084214 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d80e66f0 con 0x7f05d8047c50 2023-03-03T06:25:46.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f607cff9700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6064064f30 0x7f60640673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7fa202d4a700 1 -- 172.21.15.3:0/4053674556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1fc0cf230 msgr2=0x7fa1fc158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7fa202d4a700 1 --2- 172.21.15.3:0/4053674556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1fc0cf230 0x7fa1fc158310 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009dc0 tx=0x7fa1f400b330 comp rx=0 tx=0).stop 2023-03-03T06:25:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f607cff9700 1 -- 172.21.15.3:0/1208361213 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6074082750 con 0x7f607803b0f0 2023-03-03T06:25:46.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f607f7fe700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6064064f30 0x7f60640673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f67eb7fe700 1 -- 172.21.15.3:0/1028386761 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f67f8019030 con 0x7f6804070a00 2023-03-03T06:25:46.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e506a700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d8047c50 0x7f05d8185120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f953bbba700 1 Processor -- start 2023-03-03T06:25:46.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7fa202d4a700 1 -- 172.21.15.3:0/4053674556 shutdown_connections 2023-03-03T06:25:46.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7fa202d4a700 1 --2- 172.21.15.3:0/4053674556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1fc0cf230 0x7fa1fc158310 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7fa202d4a700 1 -- 172.21.15.3:0/4053674556 >> 172.21.15.3:0/4053674556 conn(0x7fa1fc06a030 msgr2=0x7fa1fc06a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e586b700 1 -- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d8047c50 msgr2=0x7f05d8185120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e586b700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d8047c50 0x7f05d8185120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e586b700 1 -- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05d80e6f20 msgr2=0x7f05d8039cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e586b700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05d80e6f20 0x7f05d8039cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e586b700 1 -- 172.21.15.3:0/1253084214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc0094b0 con 0x7f05d8005fa0 2023-03-03T06:25:46.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f953bbba700 1 -- start start 2023-03-03T06:25:46.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7fa202d4a700 1 -- 172.21.15.3:0/4053674556 shutdown_connections 2023-03-03T06:25:46.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.047+0000 7f05e586b700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05d8005fa0 0x7f05d80e61b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f05d000e930 tx=0x7f05d000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f67eb7fe700 1 --2- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f67f0064ee0 0x7f67f0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f67eb7fe700 1 -- 172.21.15.3:0/1028386761 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f67f8081920 con 0x7f6804070a00 2023-03-03T06:25:46.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f953bbba700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95340adec0 0x7f9534163670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f05d67fc700 1 -- 172.21.15.3:0/1253084214 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d00096c0 con 0x7f05d8005fa0 2023-03-03T06:25:46.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f05d67fc700 1 -- 172.21.15.3:0/1253084214 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d000c850 con 0x7f05d8005fa0 2023-03-03T06:25:46.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f05d67fc700 1 -- 172.21.15.3:0/1253084214 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d000cc00 con 0x7f05d8005fa0 2023-03-03T06:25:46.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f05e686d700 1 -- 172.21.15.3:0/1253084214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d80532a0 con 0x7f05d8005fa0 2023-03-03T06:25:46.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f953bbba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95340aea70 0x7f95340c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f953bbba700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9534160880 0x7f9534202550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f953bbba700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95340c6f40 con 0x7f9534160880 2023-03-03T06:25:46.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f607f7fe700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6064064f30 0x7f60640673e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60780dea00 tx=0x7f606c007d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f953bbba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95340c6c60 con 0x7f95340aea70 2023-03-03T06:25:46.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f9539956700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95340adec0 0x7f9534163670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f953bbba700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9534160df0 con 0x7f95340adec0 2023-03-03T06:25:46.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f9539956700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95340adec0 0x7f9534163670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:55458/0 (socket says 172.21.15.3:55458) 2023-03-03T06:25:46.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f9539956700 1 -- 172.21.15.3:0/3333388139 learned_addr learned my addr 172.21.15.3:0/3333388139 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:46.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7fa202d4a700 1 -- 172.21.15.3:0/4053674556 wait complete. 2023-03-03T06:25:46.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f05e686d700 1 -- 172.21.15.3:0/1253084214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d8052fc0 con 0x7f05d8005fa0 2023-03-03T06:25:46.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f9539155700 1 --2- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95340aea70 0x7f95340c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f6802ffd700 1 --2- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f67f0064ee0 0x7f67f0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f05e686d700 1 -- 172.21.15.3:0/1253084214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d804d360 con 0x7f05d8005fa0 2023-03-03T06:25:46.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f9539956700 1 -- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95340aea70 msgr2=0x7f95340c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f9539956700 1 --2- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95340aea70 0x7f95340c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f9539956700 1 -- 172.21.15.3:0/3333388139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9534160880 msgr2=0x7f9534202550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f9539956700 1 --2- 172.21.15.3:0/3333388139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9534160880 0x7f9534202550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f9539956700 1 -- 172.21.15.3:0/3333388139 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c009a70 con 0x7f95340adec0 2023-03-03T06:25:46.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f6802ffd700 1 --2- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f67f0064ee0 0x7f67f0067390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67fc000fb0 tx=0x7f67fc011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f9539956700 1 --2- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95340adec0 0x7f9534163670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95340b48c0 tx=0x7f952c004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f607cff9700 1 -- 172.21.15.3:0/1208361213 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f60740540d0 con 0x7f607803b0f0 2023-03-03T06:25:46.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f952a7fc700 1 -- 172.21.15.3:0/3333388139 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c004680 con 0x7f95340adec0 2023-03-03T06:25:46.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f952a7fc700 1 -- 172.21.15.3:0/3333388139 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952c004800 con 0x7f95340adec0 2023-03-03T06:25:46.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f952a7fc700 1 -- 172.21.15.3:0/3333388139 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c01c680 con 0x7f95340adec0 2023-03-03T06:25:46.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f953bbba700 1 -- 172.21.15.3:0/3333388139 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95340cde20 con 0x7f95340adec0 2023-03-03T06:25:46.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7faaf8880700 1 -- 172.21.15.3:0/3255647801 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec00ccf0 con 0x7faaf406ecd0 2023-03-03T06:25:46.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7faaf8880700 1 -- 172.21.15.3:0/3255647801 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaec00ce50 con 0x7faaf406ecd0 2023-03-03T06:25:46.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7faaf8880700 1 -- 172.21.15.3:0/3255647801 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec011650 con 0x7faaf406ecd0 2023-03-03T06:25:46.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7faafbae6700 1 -- 172.21.15.3:0/3255647801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf406ecd0 msgr2=0x7faaf40c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7faafbae6700 1 --2- 172.21.15.3:0/3255647801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf406ecd0 0x7faaf40c9950 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faaec0025e0 tx=0x7faaec00c190 comp rx=0 tx=0).stop 2023-03-03T06:25:46.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f42e4e62700 1 -- 172.21.15.3:0/3848422445 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d0004390 con 0x7f42d8037ff0 2023-03-03T06:25:46.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.051+0000 7f42e4e62700 1 -- 172.21.15.3:0/3848422445 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00036c0 con 0x7f42d8037ff0 2023-03-03T06:25:46.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7f42e80c8700 1 -- 172.21.15.3:0/3848422445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d8037ff0 msgr2=0x7f42d804f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7f42e80c8700 1 --2- 172.21.15.3:0/3848422445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d8037ff0 0x7f42d804f380 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f42d804bd90 tx=0x7f42d0009bf0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7f953bbba700 1 -- 172.21.15.3:0/3333388139 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95340cdb40 con 0x7f95340adec0 2023-03-03T06:25:46.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7f42e80c8700 1 -- 172.21.15.3:0/3848422445 shutdown_connections 2023-03-03T06:25:46.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7f42e80c8700 1 --2- 172.21.15.3:0/3848422445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d8037ff0 0x7f42d804f380 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7f42e80c8700 1 --2- 172.21.15.3:0/3848422445 >> v2:172.21.15.3:3300/0 conn(0x7f42d800c310 0x7f42d80367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7f42e80c8700 1 -- 172.21.15.3:0/3848422445 >> 172.21.15.3:0/3848422445 conn(0x7f42d80517b0 msgr2=0x7f42d8051b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7f67eb7fe700 1 -- 172.21.15.3:0/1028386761 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f67f804f810 con 0x7f6804070a00 2023-03-03T06:25:46.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7fa202d4a700 1 Processor -- start 2023-03-03T06:25:46.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7fa202d4a700 1 -- start start 2023-03-03T06:25:46.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7faafbae6700 1 -- 172.21.15.3:0/3255647801 shutdown_connections 2023-03-03T06:25:46.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7faafbae6700 1 --2- 172.21.15.3:0/3255647801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf406ecd0 0x7faaf40c9950 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7faafbae6700 1 -- 172.21.15.3:0/3255647801 >> 172.21.15.3:0/3255647801 conn(0x7faaf4070a80 msgr2=0x7faaf4070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7f42e80c8700 1 -- 172.21.15.3:0/3848422445 shutdown_connections 2023-03-03T06:25:46.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7f42e80c8700 1 -- 172.21.15.3:0/3848422445 wait complete. 2023-03-03T06:25:46.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fa202d4a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1fc066cc0 0x7fa1fc14e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fbb372bc700 1 Processor -- start 2023-03-03T06:25:46.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fa202d4a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1fc06d390 0x7fa1fc164730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7f05d67fc700 1 -- 172.21.15.3:0/1253084214 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f05d0019030 con 0x7f05d8005fa0 2023-03-03T06:25:46.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7f05d67fc700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f05c4064f30 0x7f05c40673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.055+0000 7f05d67fc700 1 -- 172.21.15.3:0/1253084214 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f05d0082640 con 0x7f05d8005fa0 2023-03-03T06:25:46.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7faafbae6700 1 -- 172.21.15.3:0/3255647801 shutdown_connections 2023-03-03T06:25:46.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fbb372bc700 1 -- start start 2023-03-03T06:25:46.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7f42e80c8700 1 Processor -- start 2023-03-03T06:25:46.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fa1fbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1fc06d390 0x7fa1fc164730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fa1fbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1fc06d390 0x7fa1fc164730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43540/0 (socket says 172.21.15.3:43540) 2023-03-03T06:25:46.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fa200ae6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1fc066cc0 0x7fa1fc14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fa200ae6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1fc066cc0 0x7fa1fc14e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:55468/0 (socket says 172.21.15.3:55468) 2023-03-03T06:25:46.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fa1fbfff700 1 -- 172.21.15.3:0/2013839252 learned_addr learned my addr 172.21.15.3:0/2013839252 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:46.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fa202d4a700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1fc0cf230 0x7fa1fc164c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fa202d4a700 1 -- 172.21.15.3:0/2013839252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc155910 con 0x7fa1fc06d390 2023-03-03T06:25:46.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fa202d4a700 1 -- 172.21.15.3:0/2013839252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0c5600 con 0x7fa1fc0cf230 2023-03-03T06:25:46.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fa202d4a700 1 -- 172.21.15.3:0/2013839252 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0c3a40 con 0x7fa1fc066cc0 2023-03-03T06:25:46.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7fa2012e7700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1fc0cf230 0x7fa1fc164c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7f42e80c8700 1 -- start start 2023-03-03T06:25:46.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7f42e80c8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42d800c310 0x7f42d80e33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7faafbae6700 1 -- 172.21.15.3:0/3255647801 wait complete. 2023-03-03T06:25:46.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e80c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d8037b80 0x7f42d80d9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.059+0000 7f953bbba700 1 -- 172.21.15.3:0/3333388139 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95340ca300 con 0x7f95340adec0 2023-03-03T06:25:46.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f05e506a700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f05c4064f30 0x7f05c40673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fa1fbfff700 1 -- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1fc066cc0 msgr2=0x7fa1fc14e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fa1fbfff700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1fc066cc0 0x7fa1fc14e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fa1fbfff700 1 -- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1fc0cf230 msgr2=0x7fa1fc164c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fa1fbfff700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1fc0cf230 0x7fa1fc164c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fa1fbfff700 1 -- 172.21.15.3:0/2013839252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f4009a70 con 0x7fa1fc06d390 2023-03-03T06:25:46.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e5e64700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42d800c310 0x7f42d80e33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e5e64700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42d800c310 0x7f42d80e33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:55478/0 (socket says 172.21.15.3:55478) 2023-03-03T06:25:46.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fbb372bc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb301589e0 0x7fbb300ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fa1fbfff700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1fc06d390 0x7fa1fc164730 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800c510 tx=0x7fa1e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fa2012e7700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1fc0cf230 0x7fa1fc164c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:25:46.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fbb372bc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb300c9150 con 0x7fbb30158610 2023-03-03T06:25:46.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fbb372bc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb300c99f0 con 0x7fbb300cee70 2023-03-03T06:25:46.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e5663700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d8037b80 0x7f42d80d9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e5663700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d8037b80 0x7f42d80d9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43556/0 (socket says 172.21.15.3:43556) 2023-03-03T06:25:46.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f952a7fc700 1 -- 172.21.15.3:0/3333388139 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f952c02c070 con 0x7f95340adec0 2023-03-03T06:25:46.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fbb35058700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbb30158610 0x7fbb300c9e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33738/0 (socket says 172.21.15.3:33738) 2023-03-03T06:25:46.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fbb35058700 1 -- 172.21.15.3:0/764459931 learned_addr learned my addr 172.21.15.3:0/764459931 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:46.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fbb372bc700 1 -- 172.21.15.3:0/764459931 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3015d030 con 0x7fbb301589e0 2023-03-03T06:25:46.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e80c8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d8037ff0 0x7f42d8054a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e80c8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80e5020 con 0x7f42d8037b80 2023-03-03T06:25:46.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e80c8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d804efa0 con 0x7f42d8037ff0 2023-03-03T06:25:46.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e80c8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80e3d50 con 0x7f42d800c310 2023-03-03T06:25:46.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e5e64700 1 -- 172.21.15.3:0/3021804528 learned_addr learned my addr 172.21.15.3:0/3021804528 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:46.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fa1f97fa700 1 -- 172.21.15.3:0/2013839252 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8005d90 con 0x7fa1fc06d390 2023-03-03T06:25:46.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fa1f97fa700 1 -- 172.21.15.3:0/2013839252 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e80070d0 con 0x7fa1fc06d390 2023-03-03T06:25:46.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e6e66700 1 --2- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d8037ff0 0x7f42d8054a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fa202d4a700 1 -- 172.21.15.3:0/2013839252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1fc0b1e80 con 0x7fa1fc06d390 2023-03-03T06:25:46.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7faafbae6700 1 Processor -- start 2023-03-03T06:25:46.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f05d67fc700 1 -- 172.21.15.3:0/1253084214 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f05d0050590 con 0x7f05d8005fa0 2023-03-03T06:25:46.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e6e66700 1 -- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42d800c310 msgr2=0x7f42d80e33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e6e66700 1 --2- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42d800c310 0x7f42d80e33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e6e66700 1 -- 172.21.15.3:0/3021804528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d8037b80 msgr2=0x7f42d80d9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e6e66700 1 --2- 172.21.15.3:0/3021804528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d8037b80 0x7f42d80d9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e6e66700 1 -- 172.21.15.3:0/3021804528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d0009570 con 0x7f42d8037ff0 2023-03-03T06:25:46.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7faafbae6700 1 -- start start 2023-03-03T06:25:46.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f42e6e66700 1 --2- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d8037ff0 0x7f42d8054a50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f42d0015000 tx=0x7f42d0004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7f952a7fc700 1 --2- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9518064f10 0x7f95180673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fa200ae6700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1fc066cc0 0x7fa1fc14e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:25:46.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.063+0000 7fbb34857700 1 --2- 172.21.15.3:0/764459931 >> v2:172.21.15.32:3300/0 conn(0x7fbb301589e0 0x7fbb300ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7f42d67fc700 1 -- 172.21.15.3:0/3021804528 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0003e40 con 0x7f42d8037ff0 2023-03-03T06:25:46.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7f42d67fc700 1 -- 172.21.15.3:0/3021804528 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d001f070 con 0x7f42d8037ff0 2023-03-03T06:25:46.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7f42e80c8700 1 -- 172.21.15.3:0/3021804528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d8053ea0 con 0x7f42d8037ff0 2023-03-03T06:25:46.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7f42d67fc700 1 -- 172.21.15.3:0/3021804528 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000b880 con 0x7f42d8037ff0 2023-03-03T06:25:46.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7f05e506a700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f05c4064f30 0x7f05c40673e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f05e0063e00 tx=0x7f05e0072040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7f42e80c8700 1 -- 172.21.15.3:0/3021804528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d8053270 con 0x7f42d8037ff0 2023-03-03T06:25:46.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7f9539155700 1 --2- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9518064f10 0x7f95180673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7fa202d4a700 1 -- 172.21.15.3:0/2013839252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc0b1ba0 con 0x7fa1fc06d390 2023-03-03T06:25:46.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7fbb34857700 1 -- 172.21.15.3:0/764459931 >> v1:172.21.15.32:6789/0 conn(0x7fbb300cee70 legacy=0x7fbb3015c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7fbb34857700 1 -- 172.21.15.3:0/764459931 >> v1:172.21.15.3:6789/0 conn(0x7fbb30158610 legacy=0x7fbb300c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7fbb34857700 1 -- 172.21.15.3:0/764459931 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb300c93a0 con 0x7fbb301589e0 2023-03-03T06:25:46.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7fbb34857700 1 --2- 172.21.15.3:0/764459931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb301589e0 0x7fbb300ce930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009260 tx=0x7fbb20009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a79e2b0ce3c2d1e2 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7faafbae6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaf406ecd0 0x7faaf4201780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7faaf9882700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaf406ecd0 0x7faaf4201780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7faaf9882700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaf406ecd0 0x7faaf4201780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43564/0 (socket says 172.21.15.3:43564) 2023-03-03T06:25:46.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7faaf9882700 1 -- 172.21.15.3:0/2398622789 learned_addr learned my addr 172.21.15.3:0/2398622789 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:46.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7faafbae6700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf40c9e90 0x7faaf406c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7faafbae6700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf4163ca0 0x7faaf406c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7faafbae6700 1 -- 172.21.15.3:0/2398622789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf41fef10 con 0x7faaf406ecd0 2023-03-03T06:25:46.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7faafbae6700 1 -- 172.21.15.3:0/2398622789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf41ff1f0 con 0x7faaf40c9e90 2023-03-03T06:25:46.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7f42c3fff700 1 -- 172.21.15.3:0/3021804528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d8054030 con 0x7f42d8037ff0 2023-03-03T06:25:46.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7fa1f97fa700 1 -- 172.21.15.3:0/2013839252 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8007480 con 0x7fa1fc06d390 2023-03-03T06:25:46.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7faafbae6700 1 -- 172.21.15.3:0/2398622789 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf41ff4d0 con 0x7faaf4163ca0 2023-03-03T06:25:46.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7fa1f97fa700 1 -- 172.21.15.3:0/2013839252 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa1e8019030 con 0x7fa1fc06d390 2023-03-03T06:25:46.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7f952a7fc700 1 -- 172.21.15.3:0/3333388139 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f952c083970 con 0x7f95340adec0 2023-03-03T06:25:46.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7f9539155700 1 --2- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9518064f10 0x7f95180673c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f95340cb7f0 tx=0x7f9524009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7fa202d4a700 1 -- 172.21.15.3:0/2013839252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc005310 con 0x7fa1fc06d390 2023-03-03T06:25:46.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.075+0000 7fa1f97fa700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa1ec0651c0 0x7fa1ec067670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.075+0000 7fa1f97fa700 1 -- 172.21.15.3:0/2013839252 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa1e8081480 con 0x7fa1fc06d390 2023-03-03T06:25:46.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7fbb27fff700 1 -- 172.21.15.3:0/764459931 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20010800 con 0x7fbb301589e0 2023-03-03T06:25:46.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7fbb27fff700 1 -- 172.21.15.3:0/764459931 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb20010960 con 0x7fbb301589e0 2023-03-03T06:25:46.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7fbb372bc700 1 -- 172.21.15.3:0/764459931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb301589e0 msgr2=0x7fbb300ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7fbb372bc700 1 --2- 172.21.15.3:0/764459931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb301589e0 0x7fbb300ce930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009260 tx=0x7fbb20009570 comp rx=0 tx=0).stop 2023-03-03T06:25:46.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7fbb372bc700 1 -- 172.21.15.3:0/764459931 shutdown_connections 2023-03-03T06:25:46.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7fbb372bc700 1 --2- 172.21.15.3:0/764459931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb301589e0 0x7fbb300ce930 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7fbb372bc700 1 -- 172.21.15.3:0/764459931 >> 172.21.15.3:0/764459931 conn(0x7fbb3006ace0 msgr2=0x7fbb30070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7fbb372bc700 1 -- 172.21.15.3:0/764459931 shutdown_connections 2023-03-03T06:25:46.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7fbb372bc700 1 -- 172.21.15.3:0/764459931 wait complete. 2023-03-03T06:25:46.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.067+0000 7faaf9081700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf40c9e90 0x7faaf406c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7faafa884700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf4163ca0 0x7faaf406c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7faaf9882700 1 -- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf4163ca0 msgr2=0x7faaf406c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7faaf9882700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf4163ca0 0x7faaf406c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7faaf9882700 1 -- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf40c9e90 msgr2=0x7faaf406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7faaf9882700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf40c9e90 0x7faaf406c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7faaf9882700 1 -- 172.21.15.3:0/2398622789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec007920 con 0x7faaf406ecd0 2023-03-03T06:25:46.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7faafa884700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf4163ca0 0x7faaf406c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:46.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.071+0000 7faaf9882700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaf406ecd0 0x7faaf4201780 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faaf000c580 tx=0x7faaf000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.075+0000 7fa200ae6700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa1ec0651c0 0x7fa1ec067670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.075+0000 7fbb372bc700 1 Processor -- start 2023-03-03T06:25:46.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.075+0000 7fa200ae6700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa1ec0651c0 0x7fa1ec067670 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0006fd0 tx=0x7fa1f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.075+0000 7fbb372bc700 1 -- start start 2023-03-03T06:25:46.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.075+0000 7f42d67fc700 1 -- 172.21.15.3:0/3021804528 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f42d002a020 con 0x7f42d8037ff0 2023-03-03T06:25:46.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.075+0000 7f42d67fc700 1 --2- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f42c4064fb0 0x7f42c4067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.075+0000 7f42d67fc700 1 -- 172.21.15.3:0/3021804528 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f42d0083d90 con 0x7f42d8037ff0 2023-03-03T06:25:46.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.075+0000 7f952a7fc700 1 -- 172.21.15.3:0/3333388139 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f952c051820 con 0x7f95340adec0 2023-03-03T06:25:46.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.075+0000 7f42e5e64700 1 --2- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f42c4064fb0 0x7f42c4067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb372bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb300cee70 0x7fbb30151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7f42e5e64700 1 --2- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f42c4064fb0 0x7f42c4067460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f42e0069680 tx=0x7f42e006c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb372bc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb30158610 0x7fbb300adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb372bc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb301589e0 0x7fbb3015c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb372bc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb30155f30 con 0x7fbb30158610 2023-03-03T06:25:46.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb372bc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb30156210 con 0x7fbb300cee70 2023-03-03T06:25:46.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb372bc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb301564f0 con 0x7fbb301589e0 2023-03-03T06:25:46.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb35058700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb300cee70 0x7fbb30151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb35058700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb300cee70 0x7fbb30151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39930/0 (socket says 172.21.15.3:39930) 2023-03-03T06:25:46.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb35058700 1 -- 172.21.15.3:0/1530817728 learned_addr learned my addr 172.21.15.3:0/1530817728 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:46.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb3605a700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb301589e0 0x7fbb3015c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb34857700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb30158610 0x7fbb300adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb35058700 1 -- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb301589e0 msgr2=0x7fbb3015c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb35058700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb301589e0 0x7fbb3015c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb35058700 1 -- 172.21.15.3:0/1530817728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb30158610 msgr2=0x7fbb300adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb35058700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb30158610 0x7fbb300adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7f42d67fc700 1 -- 172.21.15.3:0/3021804528 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f42d0051bd0 con 0x7f42d8037ff0 2023-03-03T06:25:46.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb35058700 1 -- 172.21.15.3:0/1530817728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb20009d40 con 0x7fbb300cee70 2023-03-03T06:25:46.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb35058700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb300cee70 0x7fbb30151350 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00c700 tx=0x7fbb2c00ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7faaea7fc700 1 -- 172.21.15.3:0/2398622789 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0004ca0 con 0x7faaf406ecd0 2023-03-03T06:25:46.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7faafbae6700 1 -- 172.21.15.3:0/2398622789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf4164bb0 con 0x7faaf406ecd0 2023-03-03T06:25:46.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7faafbae6700 1 -- 172.21.15.3:0/2398622789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf41648d0 con 0x7faaf406ecd0 2023-03-03T06:25:46.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb25ffb700 1 -- 172.21.15.3:0/1530817728 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c004ca0 con 0x7fbb300cee70 2023-03-03T06:25:46.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb25ffb700 1 -- 172.21.15.3:0/1530817728 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c004e00 con 0x7fbb300cee70 2023-03-03T06:25:46.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.079+0000 7fbb25ffb700 1 -- 172.21.15.3:0/1530817728 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c007310 con 0x7fbb300cee70 2023-03-03T06:25:46.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.083+0000 7faaea7fc700 1 -- 172.21.15.3:0/2398622789 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf00012d0 con 0x7faaf406ecd0 2023-03-03T06:25:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.083+0000 7faaea7fc700 1 -- 172.21.15.3:0/2398622789 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf00070d0 con 0x7faaf406ecd0 2023-03-03T06:25:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.083+0000 7fbb372bc700 1 -- 172.21.15.3:0/1530817728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb300cf2b0 con 0x7fbb300cee70 2023-03-03T06:25:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.083+0000 7fbb372bc700 1 -- 172.21.15.3:0/1530817728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb300c9680 con 0x7fbb300cee70 2023-03-03T06:25:46.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.083+0000 7fbb372bc700 1 -- 172.21.15.3:0/1530817728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb30071540 con 0x7fbb300cee70 2023-03-03T06:25:46.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.087+0000 7fa1f97fa700 1 -- 172.21.15.3:0/2013839252 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa1e8086060 con 0x7fa1fc06d390 2023-03-03T06:25:46.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.087+0000 7fbb25ffb700 1 -- 172.21.15.3:0/1530817728 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fbb2c01a020 con 0x7fbb300cee70 2023-03-03T06:25:46.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.091+0000 7fbb25ffb700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbb14064f30 0x7fbb140673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.091+0000 7fbb25ffb700 1 -- 172.21.15.3:0/1530817728 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbb2c05b960 con 0x7fbb300cee70 2023-03-03T06:25:46.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.091+0000 7fbb34857700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbb14064f30 0x7fbb140673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.091+0000 7fbb34857700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbb14064f30 0x7fbb140673e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb30158dd0 tx=0x7fbb20011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.087+0000 7faafbae6700 1 -- 172.21.15.3:0/2398622789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faadc005310 con 0x7faaf406ecd0 2023-03-03T06:25:46.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.087+0000 7faaea7fc700 1 -- 172.21.15.3:0/2398622789 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7faaf001a020 con 0x7faaf406ecd0 2023-03-03T06:25:46.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.091+0000 7faaea7fc700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7faad8064f10 0x7faad80673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:46.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.091+0000 7faaea7fc700 1 -- 172.21.15.3:0/2398622789 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faaf0081d10 con 0x7faaf406ecd0 2023-03-03T06:25:46.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.091+0000 7faaf9081700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7faad8064f10 0x7faad80673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:46.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.095+0000 7faaf9081700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7faad8064f10 0x7faad80673c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faaf406f0c0 tx=0x7faaec01e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:46.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.099+0000 7faaea7fc700 1 -- 172.21.15.3:0/2398622789 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7faaf004f9e0 con 0x7faaf406ecd0 2023-03-03T06:25:46.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.099+0000 7fbb25ffb700 1 -- 172.21.15.3:0/1530817728 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fbb2c050340 con 0x7fbb300cee70 2023-03-03T06:25:46.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.395+0000 7f05e686d700 1 -- 172.21.15.3:0/1253084214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f05d804d360 con 0x7f05d8005fa0 2023-03-03T06:25:46.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.395+0000 7f60867c6700 1 -- 172.21.15.3:0/1208361213 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f60780dd500 con 0x7f607803b0f0 2023-03-03T06:25:46.401 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2023-03-03T06:25:46.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.395+0000 7f05d67fc700 1 -- 172.21.15.3:0/1253084214 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f05d0017020 con 0x7f05d8005fa0 2023-03-03T06:25:46.402 INFO:teuthology.orchestra.run.smithi003.stdout:55834574857 2023-03-03T06:25:46.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.395+0000 7f607cff9700 1 -- 172.21.15.3:0/1208361213 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f60780dd500 con 0x7f607803b0f0 2023-03-03T06:25:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f05e686d700 1 -- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f05c4064f30 msgr2=0x7f05c40673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f05e686d700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f05c4064f30 0x7f05c40673e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f05e0063e00 tx=0x7f05e0072040 comp rx=0 tx=0).stop 2023-03-03T06:25:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f05e686d700 1 -- 172.21.15.3:0/1253084214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05d8005fa0 msgr2=0x7f05d80e61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f05e686d700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05d8005fa0 0x7f05d80e61b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f05d000e930 tx=0x7f05d000ecf0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f05e686d700 1 -- 172.21.15.3:0/1253084214 shutdown_connections 2023-03-03T06:25:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f05e686d700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f05c4064f30 0x7f05c40673e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f05e686d700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05d80e6f20 0x7f05d8039cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f05e686d700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f05d8047c50 0x7f05d8185120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f05e686d700 1 --2- 172.21.15.3:0/1253084214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f05d8005fa0 0x7f05d80e61b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f05e686d700 1 -- 172.21.15.3:0/1253084214 >> 172.21.15.3:0/1253084214 conn(0x7f05d8051710 msgr2=0x7f05d80e9300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f05e686d700 1 -- 172.21.15.3:0/1253084214 shutdown_connections 2023-03-03T06:25:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f05e686d700 1 -- 172.21.15.3:0/1253084214 wait complete. 2023-03-03T06:25:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f60867c6700 1 -- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6064064f30 msgr2=0x7f60640673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f60867c6700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6064064f30 0x7f60640673e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f60780dea00 tx=0x7f606c007d90 comp rx=0 tx=0).stop 2023-03-03T06:25:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f60867c6700 1 -- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f607803b0f0 msgr2=0x7f60780ddea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f60867c6700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f607803b0f0 0x7f60780ddea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f607400e890 tx=0x7f607400ec50 comp rx=0 tx=0).stop 2023-03-03T06:25:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f60867c6700 1 -- 172.21.15.3:0/1208361213 shutdown_connections 2023-03-03T06:25:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.399+0000 7f60867c6700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6064064f30 0x7f60640673e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.403+0000 7f60867c6700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60780e2970 0x7f60780e08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.403+0000 7f60867c6700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6078053520 0x7f60780dc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.403+0000 7f60867c6700 1 --2- 172.21.15.3:0/1208361213 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f607803b0f0 0x7f60780ddea0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.403+0000 7f60867c6700 1 -- 172.21.15.3:0/1208361213 >> 172.21.15.3:0/1208361213 conn(0x7f60780559e0 msgr2=0x7f6078037bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.403+0000 7f60867c6700 1 -- 172.21.15.3:0/1208361213 shutdown_connections 2023-03-03T06:25:46.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.403+0000 7f60867c6700 1 -- 172.21.15.3:0/1208361213 wait complete. 2023-03-03T06:25:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.411+0000 7f68092d8700 1 -- 172.21.15.3:0/1028386761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6804000940 con 0x7f6804070a00 2023-03-03T06:25:46.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.411+0000 7f67eb7fe700 1 -- 172.21.15.3:0/1028386761 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f67f8052e30 con 0x7f6804070a00 2023-03-03T06:25:46.418 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2023-03-03T06:25:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7fa202d4a700 1 -- 172.21.15.3:0/2013839252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa1dc0055c0 con 0x7fa1fc06d390 2023-03-03T06:25:46.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7f67e97fa700 1 -- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f67f0064ee0 msgr2=0x7f67f0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7f67e97fa700 1 --2- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f67f0064ee0 0x7f67f0067390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67fc000fb0 tx=0x7f67fc011000 comp rx=0 tx=0).stop 2023-03-03T06:25:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7f67e97fa700 1 -- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6804070a00 msgr2=0x7f68040c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7f67e97fa700 1 --2- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6804070a00 0x7f68040c9490 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f67f800c490 tx=0x7f67f800c7a0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7f67e97fa700 1 -- 172.21.15.3:0/1028386761 shutdown_connections 2023-03-03T06:25:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7f67e97fa700 1 --2- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f67f0064ee0 0x7f67f0067390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7f67e97fa700 1 --2- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f680415c8c0 0x7f680414aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7f67e97fa700 1 --2- 172.21.15.3:0/1028386761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6804070a00 0x7f68040c9490 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7f67e97fa700 1 --2- 172.21.15.3:0/1028386761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f680406e5c0 0x7f68040b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7f67e97fa700 1 -- 172.21.15.3:0/1028386761 >> 172.21.15.3:0/1028386761 conn(0x7f68040d0310 msgr2=0x7f68040b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7fa1f97fa700 1 -- 172.21.15.3:0/2013839252 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa1e8086060 con 0x7fa1fc06d390 2023-03-03T06:25:46.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7f67e97fa700 1 -- 172.21.15.3:0/1028386761 shutdown_connections 2023-03-03T06:25:46.422 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2023-03-03T06:25:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.415+0000 7f67e97fa700 1 -- 172.21.15.3:0/1028386761 wait complete. 2023-03-03T06:25:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.419+0000 7fa202d4a700 1 -- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa1ec0651c0 msgr2=0x7fa1ec067670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.419+0000 7fa202d4a700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa1ec0651c0 0x7fa1ec067670 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0006fd0 tx=0x7fa1f0008040 comp rx=0 tx=0).stop 2023-03-03T06:25:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.419+0000 7fa202d4a700 1 -- 172.21.15.3:0/2013839252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1fc06d390 msgr2=0x7fa1fc164730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.419+0000 7fa202d4a700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1fc06d390 0x7fa1fc164730 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800c510 tx=0x7fa1e800c8d0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.419+0000 7fa202d4a700 1 -- 172.21.15.3:0/2013839252 shutdown_connections 2023-03-03T06:25:46.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.419+0000 7fa202d4a700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa1ec0651c0 0x7fa1ec067670 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.419+0000 7fa202d4a700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1fc0cf230 0x7fa1fc164c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.419+0000 7fa202d4a700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1fc06d390 0x7fa1fc164730 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.419+0000 7fa202d4a700 1 --2- 172.21.15.3:0/2013839252 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa1fc066cc0 0x7fa1fc14e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.419+0000 7fa202d4a700 1 -- 172.21.15.3:0/2013839252 >> 172.21.15.3:0/2013839252 conn(0x7fa1fc06a030 msgr2=0x7fa1fc157620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.419+0000 7fa202d4a700 1 -- 172.21.15.3:0/2013839252 shutdown_connections 2023-03-03T06:25:46.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.419+0000 7fa202d4a700 1 -- 172.21.15.3:0/2013839252 wait complete. 2023-03-03T06:25:46.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.423+0000 7f953bbba700 1 -- 172.21.15.3:0/3333388139 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9534160c50 con 0x7f95340adec0 2023-03-03T06:25:46.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.423+0000 7f952a7fc700 1 -- 172.21.15.3:0/3333388139 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9534160c50 con 0x7f95340adec0 2023-03-03T06:25:46.432 INFO:teuthology.orchestra.run.smithi003.stdout:55834574857 2023-03-03T06:25:46.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.427+0000 7f42c3fff700 1 -- 172.21.15.3:0/3021804528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f42d8054030 con 0x7f42d8037ff0 2023-03-03T06:25:46.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.427+0000 7f9517fff700 1 -- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9518064f10 msgr2=0x7f95180673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.427+0000 7f42d67fc700 1 -- 172.21.15.3:0/3021804528 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f42d0010380 con 0x7f42d8037ff0 2023-03-03T06:25:46.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.427+0000 7f9517fff700 1 --2- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9518064f10 0x7f95180673c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f95340cb7f0 tx=0x7f9524009040 comp rx=0 tx=0).stop 2023-03-03T06:25:46.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.427+0000 7f9517fff700 1 -- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95340adec0 msgr2=0x7f9534163670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.427+0000 7f9517fff700 1 --2- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95340adec0 0x7f9534163670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95340b48c0 tx=0x7f952c004af0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.427+0000 7f9517fff700 1 -- 172.21.15.3:0/3333388139 shutdown_connections 2023-03-03T06:25:46.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.427+0000 7f9517fff700 1 --2- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9518064f10 0x7f95180673c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.427+0000 7f9517fff700 1 --2- 172.21.15.3:0/3333388139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9534160880 0x7f9534202550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.427+0000 7f9517fff700 1 --2- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95340aea70 0x7f95340c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.427+0000 7f9517fff700 1 --2- 172.21.15.3:0/3333388139 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f95340adec0 0x7f9534163670 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.427+0000 7f9517fff700 1 -- 172.21.15.3:0/3333388139 >> 172.21.15.3:0/3333388139 conn(0x7f9534070a30 msgr2=0x7f95340aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.435 INFO:teuthology.orchestra.run.smithi003.stdout:55834574857 2023-03-03T06:25:46.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.431+0000 7f9517fff700 1 -- 172.21.15.3:0/3333388139 shutdown_connections 2023-03-03T06:25:46.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.431+0000 7f42c3fff700 1 -- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f42c4064fb0 msgr2=0x7f42c4067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.431+0000 7f42c3fff700 1 --2- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f42c4064fb0 0x7f42c4067460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f42e0069680 tx=0x7f42e006c040 comp rx=0 tx=0).stop 2023-03-03T06:25:46.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.431+0000 7f9517fff700 1 -- 172.21.15.3:0/3333388139 wait complete. 2023-03-03T06:25:46.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.431+0000 7f42c3fff700 1 -- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d8037ff0 msgr2=0x7f42d8054a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.431+0000 7f42c3fff700 1 --2- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d8037ff0 0x7f42d8054a50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f42d0015000 tx=0x7f42d0004770 comp rx=0 tx=0).stop 2023-03-03T06:25:46.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.431+0000 7f42c3fff700 1 -- 172.21.15.3:0/3021804528 shutdown_connections 2023-03-03T06:25:46.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.431+0000 7f42c3fff700 1 --2- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f42c4064fb0 0x7f42c4067460 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.431+0000 7f42c3fff700 1 --2- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d8037ff0 0x7f42d8054a50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.431+0000 7f42c3fff700 1 --2- 172.21.15.3:0/3021804528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d8037b80 0x7f42d80d9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.431+0000 7f42c3fff700 1 --2- 172.21.15.3:0/3021804528 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42d800c310 0x7f42d80e33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.431+0000 7f42c3fff700 1 -- 172.21.15.3:0/3021804528 >> 172.21.15.3:0/3021804528 conn(0x7f42d80517b0 msgr2=0x7f42d80dd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.435+0000 7f42c3fff700 1 -- 172.21.15.3:0/3021804528 shutdown_connections 2023-03-03T06:25:46.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.435+0000 7f42c3fff700 1 -- 172.21.15.3:0/3021804528 wait complete. 2023-03-03T06:25:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.443+0000 7faafbae6700 1 -- 172.21.15.3:0/2398622789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7faadc0055c0 con 0x7faaf406ecd0 2023-03-03T06:25:46.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.443+0000 7faaea7fc700 1 -- 172.21.15.3:0/2398622789 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faaf0053000 con 0x7faaf406ecd0 2023-03-03T06:25:46.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.443+0000 7fbb372bc700 1 -- 172.21.15.3:0/1530817728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbb30156350 con 0x7fbb300cee70 2023-03-03T06:25:46.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574859 for osd.1 2023-03-03T06:25:46.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.443+0000 7fbb25ffb700 1 -- 172.21.15.3:0/1530817728 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb30156350 con 0x7fbb300cee70 2023-03-03T06:25:46.450 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2023-03-03T06:25:46.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.3 2023-03-03T06:25:46.451 INFO:teuthology.orchestra.run.smithi003.stdout:55834574858 2023-03-03T06:25:46.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7fbb372bc700 1 -- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbb14064f30 msgr2=0x7fbb140673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7fbb372bc700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbb14064f30 0x7fbb140673e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb30158dd0 tx=0x7fbb20011000 comp rx=0 tx=0).stop 2023-03-03T06:25:46.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7fbb372bc700 1 -- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb300cee70 msgr2=0x7fbb30151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7fbb372bc700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb300cee70 0x7fbb30151350 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00c700 tx=0x7fbb2c00ca10 comp rx=0 tx=0).stop 2023-03-03T06:25:46.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7fbb372bc700 1 -- 172.21.15.3:0/1530817728 shutdown_connections 2023-03-03T06:25:46.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7fbb372bc700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbb14064f30 0x7fbb140673e0 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb30158dd0 tx=0x7fbb20011000 comp rx=0 tx=0).stop 2023-03-03T06:25:46.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7fbb372bc700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb301589e0 0x7fbb3015c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7fbb372bc700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb30158610 0x7fbb300adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7fbb372bc700 1 --2- 172.21.15.3:0/1530817728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb300cee70 0x7fbb30151350 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7fbb372bc700 1 -- 172.21.15.3:0/1530817728 >> 172.21.15.3:0/1530817728 conn(0x7fbb3006ace0 msgr2=0x7fbb300c9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7fbb372bc700 1 -- 172.21.15.3:0/1530817728 shutdown_connections 2023-03-03T06:25:46.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7fbb372bc700 1 -- 172.21.15.3:0/1530817728 wait complete. 2023-03-03T06:25:46.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7faafbae6700 1 -- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7faad8064f10 msgr2=0x7faad80673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7faafbae6700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7faad8064f10 0x7faad80673c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faaf406f0c0 tx=0x7faaec01e040 comp rx=0 tx=0).stop 2023-03-03T06:25:46.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7faafbae6700 1 -- 172.21.15.3:0/2398622789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaf406ecd0 msgr2=0x7faaf4201780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:46.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.447+0000 7faafbae6700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaf406ecd0 0x7faaf4201780 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faaf000c580 tx=0x7faaf000c890 comp rx=0 tx=0).stop 2023-03-03T06:25:46.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.451+0000 7faafbae6700 1 -- 172.21.15.3:0/2398622789 shutdown_connections 2023-03-03T06:25:46.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.451+0000 7faafbae6700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7faad8064f10 0x7faad80673c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.451+0000 7faafbae6700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7faaf4163ca0 0x7faaf406c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.451+0000 7faafbae6700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaf40c9e90 0x7faaf406c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.451+0000 7faafbae6700 1 --2- 172.21.15.3:0/2398622789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaf406ecd0 0x7faaf4201780 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:46.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.451+0000 7faafbae6700 1 -- 172.21.15.3:0/2398622789 >> 172.21.15.3:0/2398622789 conn(0x7faaf4070a80 msgr2=0x7faaf4161d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:46.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.451+0000 7faafbae6700 1 -- 172.21.15.3:0/2398622789 shutdown_connections 2023-03-03T06:25:46.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:46.451+0000 7faafbae6700 1 -- 172.21.15.3:0/2398622789 wait complete. 2023-03-03T06:25:46.466 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.7 2023-03-03T06:25:46.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.0 2023-03-03T06:25:46.474 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.6 2023-03-03T06:25:46.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.2 2023-03-03T06:25:46.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.4 2023-03-03T06:25:46.486 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574858 for osd.5 2023-03-03T06:25:47.450 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-03-03T06:25:47.451 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-03-03T06:25:47.467 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-03-03T06:25:47.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-03-03T06:25:47.476 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-03-03T06:25:47.477 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-03-03T06:25:47.484 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-03-03T06:25:47.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-03-03T06:25:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.707+0000 7f1fb1a5b700 1 Processor -- start 2023-03-03T06:25:47.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.711+0000 7f1fb1a5b700 1 -- start start 2023-03-03T06:25:47.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.711+0000 7f1fb1a5b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fac15fa00 0x7f1fac0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.711+0000 7f1fb1a5b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fac161050 con 0x7f1fac0c9890 2023-03-03T06:25:47.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.711+0000 7f1fb1a5b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fac1607e0 con 0x7f1fac0add80 2023-03-03T06:25:47.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.711+0000 7f1fb1a5b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac160a90 con 0x7f1fac15fa00 2023-03-03T06:25:47.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1fab7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1fac0c9890 0x7f1fac0ac060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33742/0 (socket says 172.21.15.3:33742) 2023-03-03T06:25:47.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1fab7fe700 1 -- 172.21.15.3:0/2881467793 learned_addr learned my addr 172.21.15.3:0/2881467793 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1faaffd700 1 --2- 172.21.15.3:0/2881467793 >> v2:172.21.15.32:3300/0 conn(0x7f1fac15fa00 0x7f1fac0cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1fa9ffb700 1 -- 172.21.15.3:0/2881467793 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133183860 0 0) 0x7f1fac161050 con 0x7f1fac0c9890 2023-03-03T06:25:47.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1fa9ffb700 1 -- 172.21.15.3:0/2881467793 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f940035d0 con 0x7f1fac0c9890 2023-03-03T06:25:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1faaffd700 1 -- 172.21.15.3:0/2881467793 >> v1:172.21.15.32:6789/0 conn(0x7f1fac0add80 legacy=0x7f1fac1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1faaffd700 1 -- 172.21.15.3:0/2881467793 >> v1:172.21.15.3:6789/0 conn(0x7f1fac0c9890 legacy=0x7f1fac0ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1faaffd700 1 -- 172.21.15.3:0/2881467793 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac160d70 con 0x7f1fac15fa00 2023-03-03T06:25:47.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f7d776a7700 1 Processor -- start 2023-03-03T06:25:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1faaffd700 1 --2- 172.21.15.3:0/2881467793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fac15fa00 0x7f1fac0cdb20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009850 tx=0x7f1fa0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2012bd4edc4a8863 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f7d776a7700 1 -- start start 2023-03-03T06:25:47.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1fa9ffb700 1 -- 172.21.15.3:0/2881467793 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0004230 con 0x7f1fac15fa00 2023-03-03T06:25:47.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1fa9ffb700 1 -- 172.21.15.3:0/2881467793 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa0004390 con 0x7f1fac15fa00 2023-03-03T06:25:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1fa9ffb700 1 -- 172.21.15.3:0/2881467793 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00036c0 con 0x7f1fac15fa00 2023-03-03T06:25:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1fb1a5b700 1 -- 172.21.15.3:0/2881467793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fac15fa00 msgr2=0x7f1fac0cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.715+0000 7f1fb1a5b700 1 --2- 172.21.15.3:0/2881467793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fac15fa00 0x7f1fac0cdb20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009850 tx=0x7f1fa0009b60 comp rx=0 tx=0).stop 2023-03-03T06:25:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f1fb1a5b700 1 -- 172.21.15.3:0/2881467793 shutdown_connections 2023-03-03T06:25:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f1fb1a5b700 1 --2- 172.21.15.3:0/2881467793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fac15fa00 0x7f1fac0cdb20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f1fb1a5b700 1 -- 172.21.15.3:0/2881467793 >> 172.21.15.3:0/2881467793 conn(0x7f1fac060be0 msgr2=0x7f1fac070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f1fb1a5b700 1 -- 172.21.15.3:0/2881467793 shutdown_connections 2023-03-03T06:25:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d776a7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d700af310 0x7f7d7015e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d776a7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d700b04f0 con 0x7f7d7015d8e0 2023-03-03T06:25:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d75443700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7d7015d8e0 0x7f7d700b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33744/0 (socket says 172.21.15.3:33744) 2023-03-03T06:25:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d75443700 1 -- 172.21.15.3:0/1891485989 learned_addr learned my addr 172.21.15.3:0/1891485989 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d776a7700 1 -- 172.21.15.3:0/1891485989 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7015dcb0 con 0x7f7d700c9a60 2023-03-03T06:25:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d776a7700 1 -- 172.21.15.3:0/1891485989 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7015ee30 con 0x7f7d700af310 2023-03-03T06:25:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f1fb1a5b700 1 -- 172.21.15.3:0/2881467793 wait complete. 2023-03-03T06:25:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155997331 0 0) 0x7f7d700b04f0 con 0x7f7d7015d8e0 2023-03-03T06:25:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d5c0035d0 con 0x7f7d7015d8e0 2023-03-03T06:25:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d75c44700 1 --2- 172.21.15.3:0/1891485989 >> v2:172.21.15.32:3300/0 conn(0x7f7d700af310 0x7f7d7015e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2350853128 0 0) 0x7f7d5c0035d0 con 0x7f7d7015d8e0 2023-03-03T06:25:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 >> v2:172.21.15.32:3300/0 conn(0x7f7d700af310 msgr2=0x7f7d7015e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 --2- 172.21.15.3:0/1891485989 >> v2:172.21.15.32:3300/0 conn(0x7f7d700af310 0x7f7d7015e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 >> v1:172.21.15.32:6789/0 conn(0x7f7d700c9a60 legacy=0x7f7d700aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d700c9ee0 con 0x7f7d7015d8e0 2023-03-03T06:25:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f7d6c001670 con 0x7f7d7015d8e0 2023-03-03T06:25:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 >> v1:172.21.15.3:6789/0 conn(0x7f7d7015d8e0 legacy=0x7f7d700b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 --2- 172.21.15.3:0/1891485989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d5c003a20 0x7f7d5c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 --2- 172.21.15.3:0/1891485989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d5c004450 0x7f7d5c004910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 --2- 172.21.15.3:0/1891485989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d5c0051b0 0x7f7d5c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7015ee30 con 0x7f7d5c0051b0 2023-03-03T06:25:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c00c100 con 0x7f7d5c003a20 2023-03-03T06:25:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.719+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c00c2c0 con 0x7f7d5c004450 2023-03-03T06:25:47.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f1fb1a5b700 1 Processor -- start 2023-03-03T06:25:47.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f7d75443700 1 --2- 172.21.15.3:0/1891485989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d5c003a20 0x7f7d5c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f7d75c44700 1 --2- 172.21.15.3:0/1891485989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d5c0051b0 0x7f7d5c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f7d74c42700 1 --2- 172.21.15.3:0/1891485989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d5c004450 0x7f7d5c004910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f1fb1a5b700 1 -- start start 2023-03-03T06:25:47.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f7d74c42700 1 -- 172.21.15.3:0/1891485989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d5c003a20 msgr2=0x7f7d5c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:47.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f7d74c42700 1 --2- 172.21.15.3:0/1891485989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d5c003a20 0x7f7d5c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f7d74c42700 1 -- 172.21.15.3:0/1891485989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d5c0051b0 msgr2=0x7f7d5c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:47.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f7d74c42700 1 --2- 172.21.15.3:0/1891485989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d5c0051b0 0x7f7d5c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f7d74c42700 1 -- 172.21.15.3:0/1891485989 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d700c9ee0 con 0x7f7d5c004450 2023-03-03T06:25:47.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f7d75443700 1 --2- 172.21.15.3:0/1891485989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d5c003a20 0x7f7d5c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:47.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f7d75c44700 1 -- 172.21.15.3:0/1891485989 reap_dead start 2023-03-03T06:25:47.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f7d74c42700 1 --2- 172.21.15.3:0/1891485989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d5c004450 0x7f7d5c004910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000e850 tx=0x7f7d5c0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee30e76de093f642 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f1fb1a5b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0add80 0x7f1fac0c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f1fb1a5b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fac0c9890 0x7f1fac0c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f1fb1a5b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fac15fa00 0x7f1fac0c2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f1fb1a5b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0b1e60 con 0x7f1fac0add80 2023-03-03T06:25:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f1fb1a5b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0c48f0 con 0x7f1fac0c9890 2023-03-03T06:25:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f1fb1a5b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0c4bd0 con 0x7f1fac15fa00 2023-03-03T06:25:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f1faaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0add80 0x7f1fac0c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f1faaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0add80 0x7f1fac0c4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43600/0 (socket says 172.21.15.3:43600) 2023-03-03T06:25:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f1faaffd700 1 -- 172.21.15.3:0/122648050 learned_addr learned my addr 172.21.15.3:0/122648050 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f1faa7fc700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fac0c9890 0x7f1fac0c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60013070 con 0x7f7d5c004450 2023-03-03T06:25:47.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6000e0d0 con 0x7f7d5c004450 2023-03-03T06:25:47.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f7d67fff700 1 -- 172.21.15.3:0/1891485989 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6000e530 con 0x7f7d5c004450 2023-03-03T06:25:47.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.723+0000 7f1fab7fe700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fac15fa00 0x7f1fac0c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f1faa7fc700 1 -- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fac15fa00 msgr2=0x7f1fac0c2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f1faa7fc700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fac15fa00 0x7f1fac0c2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f1faa7fc700 1 -- 172.21.15.3:0/122648050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0add80 msgr2=0x7f1fac0c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f1faa7fc700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0add80 0x7f1fac0c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f1faa7fc700 1 -- 172.21.15.3:0/122648050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa00094b0 con 0x7f1fac0c9890 2023-03-03T06:25:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f7d776a7700 1 -- 172.21.15.3:0/1891485989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d5c004450 msgr2=0x7f7d5c004910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f7d776a7700 1 --2- 172.21.15.3:0/1891485989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d5c004450 0x7f7d5c004910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000e850 tx=0x7f7d5c0058a0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f1faa7fc700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fac0c9890 0x7f1fac0c22d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00b3c0 tx=0x7f1f9c00b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f1f93fff700 1 -- 172.21.15.3:0/122648050 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011830 con 0x7f1fac0c9890 2023-03-03T06:25:47.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f1f93fff700 1 -- 172.21.15.3:0/122648050 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c0119b0 con 0x7f1fac0c9890 2023-03-03T06:25:47.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f1f93fff700 1 -- 172.21.15.3:0/122648050 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011d60 con 0x7f1fac0c9890 2023-03-03T06:25:47.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f1fb1a5b700 1 -- 172.21.15.3:0/122648050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fac0cd600 con 0x7f1fac0c9890 2023-03-03T06:25:47.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f7d776a7700 1 -- 172.21.15.3:0/1891485989 shutdown_connections 2023-03-03T06:25:47.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f7d776a7700 1 --2- 172.21.15.3:0/1891485989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d5c004450 0x7f7d5c004910 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f7d776a7700 1 -- 172.21.15.3:0/1891485989 >> 172.21.15.3:0/1891485989 conn(0x7f7d70060be0 msgr2=0x7f7d70070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:47.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f1fb1a5b700 1 -- 172.21.15.3:0/122648050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fac0cd320 con 0x7f1fac0c9890 2023-03-03T06:25:47.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f1fb1a5b700 1 -- 172.21.15.3:0/122648050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fac15fdd0 con 0x7f1fac0c9890 2023-03-03T06:25:47.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f7d776a7700 1 -- 172.21.15.3:0/1891485989 shutdown_connections 2023-03-03T06:25:47.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.727+0000 7f7d776a7700 1 -- 172.21.15.3:0/1891485989 wait complete. 2023-03-03T06:25:47.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.731+0000 7f7d776a7700 1 Processor -- start 2023-03-03T06:25:47.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.731+0000 7f7d776a7700 1 -- start start 2023-03-03T06:25:47.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d776a7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d5c004450 0x7f7d701f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d776a7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d700b1b90 0x7f7d700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.731+0000 7f1f93fff700 1 -- 172.21.15.3:0/122648050 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1f9c021070 con 0x7f1fac0c9890 2023-03-03T06:25:47.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f1f93fff700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1f98064fb0 0x7f1f98067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f1f93fff700 1 -- 172.21.15.3:0/122648050 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1f9c081f20 con 0x7f1fac0c9890 2023-03-03T06:25:47.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d74c42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d700b1b90 0x7f7d700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d74c42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d700b1b90 0x7f7d700b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43614/0 (socket says 172.21.15.3:43614) 2023-03-03T06:25:47.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d776a7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d7015a6f0 0x7f7d7015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d776a7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d700b2540 con 0x7f7d700b1b90 2023-03-03T06:25:47.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d776a7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d701f87a0 con 0x7f7d5c004450 2023-03-03T06:25:47.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d776a7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7015b460 con 0x7f7d7015a6f0 2023-03-03T06:25:47.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d74c42700 1 -- 172.21.15.3:0/2888984364 learned_addr learned my addr 172.21.15.3:0/2888984364 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:47.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f1faaffd700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1f98064fb0 0x7f1f98067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d75c44700 1 --2- 172.21.15.3:0/2888984364 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d7015a6f0 0x7f7d7015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d74c42700 1 -- 172.21.15.3:0/2888984364 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d7015a6f0 msgr2=0x7f7d7015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d74c42700 1 --2- 172.21.15.3:0/2888984364 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d7015a6f0 0x7f7d7015ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d74c42700 1 -- 172.21.15.3:0/2888984364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d5c004450 msgr2=0x7f7d701f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d74c42700 1 --2- 172.21.15.3:0/2888984364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d5c004450 0x7f7d701f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.735+0000 7f7d74c42700 1 -- 172.21.15.3:0/2888984364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d700ca420 con 0x7f7d700b1b90 2023-03-03T06:25:47.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.739+0000 7f7d74c42700 1 --2- 172.21.15.3:0/2888984364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d700b1b90 0x7f7d700b2000 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7d60004000 tx=0x7f7d60001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.739+0000 7f1faaffd700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1f98064fb0 0x7f1f98067460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009ba0 tx=0x7f1fa0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.739+0000 7f7d65ffb700 1 -- 172.21.15.3:0/2888984364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60003ce0 con 0x7f7d700b1b90 2023-03-03T06:25:47.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.739+0000 7f7d65ffb700 1 -- 172.21.15.3:0/2888984364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d60003e40 con 0x7f7d700b1b90 2023-03-03T06:25:47.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.739+0000 7f7d65ffb700 1 -- 172.21.15.3:0/2888984364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60006ac0 con 0x7f7d700b1b90 2023-03-03T06:25:47.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.739+0000 7f7d776a7700 1 -- 172.21.15.3:0/2888984364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d700ca140 con 0x7f7d700b1b90 2023-03-03T06:25:47.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.739+0000 7f7d776a7700 1 -- 172.21.15.3:0/2888984364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d700ca700 con 0x7f7d700b1b90 2023-03-03T06:25:47.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.743+0000 7f7d776a7700 1 -- 172.21.15.3:0/2888984364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d70157020 con 0x7f7d700b1b90 2023-03-03T06:25:47.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.743+0000 7f7d65ffb700 1 -- 172.21.15.3:0/2888984364 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7d60026020 con 0x7f7d700b1b90 2023-03-03T06:25:47.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.743+0000 7f1f93fff700 1 -- 172.21.15.3:0/122648050 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1f9c04fd60 con 0x7f1fac0c9890 2023-03-03T06:25:47.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.743+0000 7f7d65ffb700 1 --2- 172.21.15.3:0/2888984364 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d54064f30 0x7f7d540673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.743+0000 7f7d65ffb700 1 -- 172.21.15.3:0/2888984364 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7d60082250 con 0x7f7d700b1b90 2023-03-03T06:25:47.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.751+0000 7f7d75443700 1 --2- 172.21.15.3:0/2888984364 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d54064f30 0x7f7d540673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.751+0000 7f7d75443700 1 --2- 172.21.15.3:0/2888984364 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d54064f30 0x7f7d540673e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000fb0 tx=0x7f7d6c001b70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.751+0000 7f7d65ffb700 1 -- 172.21.15.3:0/2888984364 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7d60050150 con 0x7f7d700b1b90 2023-03-03T06:25:47.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.779+0000 7f5d7b29f700 1 Processor -- start 2023-03-03T06:25:47.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.779+0000 7f5d7b29f700 1 -- start start 2023-03-03T06:25:47.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.779+0000 7f5d7b29f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d741608d0 0x7f5d740cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.779+0000 7f5d7b29f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5d740cfb60 0x7f5d740d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.779+0000 7f5d7b29f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d74160440 con 0x7f5d74153670 2023-03-03T06:25:47.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.779+0000 7f5d7b29f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d740cffa0 con 0x7f5d741608d0 2023-03-03T06:25:47.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d7b29f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d740cd3e0 con 0x7f5d740cfb60 2023-03-03T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d79a9c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d741608d0 0x7f5d740cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d79a9c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d741608d0 0x7f5d740cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43630/0 (socket says 172.21.15.3:43630) 2023-03-03T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d79a9c700 1 -- 172.21.15.3:0/4069965521 learned_addr learned my addr 172.21.15.3:0/4069965521 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d7aa9e700 1 --2- 172.21.15.3:0/4069965521 >> v2:172.21.15.32:3300/0 conn(0x7f5d740cfb60 0x7f5d740d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d79a9c700 1 -- 172.21.15.3:0/4069965521 >> v2:172.21.15.32:3300/0 conn(0x7f5d740cfb60 msgr2=0x7f5d740d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:47.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d79a9c700 1 --2- 172.21.15.3:0/4069965521 >> v2:172.21.15.32:3300/0 conn(0x7f5d740cfb60 0x7f5d740d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d79a9c700 1 -- 172.21.15.3:0/4069965521 >> v1:172.21.15.32:6789/0 conn(0x7f5d74153670 legacy=0x7f5d74071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d79a9c700 1 -- 172.21.15.3:0/4069965521 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d740cd6c0 con 0x7f5d741608d0 2023-03-03T06:25:47.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d79a9c700 1 --2- 172.21.15.3:0/4069965521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d741608d0 0x7f5d740cf620 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009910 tx=0x7f5d64009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205e2e9b0d136e4e server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d7929b700 1 -- 172.21.15.3:0/4069965521 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6400ba20 con 0x7f5d741608d0 2023-03-03T06:25:47.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d7929b700 1 -- 172.21.15.3:0/4069965521 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6400bb80 con 0x7f5d741608d0 2023-03-03T06:25:47.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d7929b700 1 -- 172.21.15.3:0/4069965521 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d64009e60 con 0x7f5d741608d0 2023-03-03T06:25:47.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f18256a4700 1 Processor -- start 2023-03-03T06:25:47.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d7b29f700 1 -- 172.21.15.3:0/4069965521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d741608d0 msgr2=0x7f5d740cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f5d7b29f700 1 --2- 172.21.15.3:0/4069965521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d741608d0 0x7f5d740cf620 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009910 tx=0x7f5d64009c20 comp rx=0 tx=0).stop 2023-03-03T06:25:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.783+0000 7f18256a4700 1 -- start start 2023-03-03T06:25:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.787+0000 7f5d7b29f700 1 -- 172.21.15.3:0/4069965521 shutdown_connections 2023-03-03T06:25:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.787+0000 7f5d7b29f700 1 --2- 172.21.15.3:0/4069965521 >> v2:172.21.15.32:3300/0 conn(0x7f5d740cfb60 0x7f5d740d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.787+0000 7f5d7b29f700 1 --2- 172.21.15.3:0/4069965521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d741608d0 0x7f5d740cf620 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.787+0000 7f5d7b29f700 1 -- 172.21.15.3:0/4069965521 >> 172.21.15.3:0/4069965521 conn(0x7f5d7406e5c0 msgr2=0x7f5d7406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.787+0000 7f5d7b29f700 1 -- 172.21.15.3:0/4069965521 shutdown_connections 2023-03-03T06:25:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.787+0000 7f5d7b29f700 1 -- 172.21.15.3:0/4069965521 wait complete. 2023-03-03T06:25:47.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.787+0000 7f5d7b29f700 1 Processor -- start 2023-03-03T06:25:47.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.787+0000 7f18256a4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1820153670 0x7f1820071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.787+0000 7f18256a4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18201608d0 0x7f18200cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f18256a4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1820160440 con 0x7f1820160d20 2023-03-03T06:25:47.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f181ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1820153670 0x7f1820071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f181ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1820153670 0x7f1820071350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:55518/0 (socket says 172.21.15.3:55518) 2023-03-03T06:25:47.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f181ffff700 1 -- 172.21.15.3:0/2501441099 learned_addr learned my addr 172.21.15.3:0/2501441099 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:47.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f18256a4700 1 -- 172.21.15.3:0/2501441099 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18200cd3e0 con 0x7f18201608d0 2023-03-03T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f18256a4700 1 -- 172.21.15.3:0/2501441099 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18200cd6c0 con 0x7f1820153670 2023-03-03T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f181f7fe700 1 --2- 172.21.15.3:0/2501441099 >> v2:172.21.15.32:3300/0 conn(0x7f18201608d0 0x7f18200cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f181ffff700 1 -- 172.21.15.3:0/2501441099 >> v2:172.21.15.32:3300/0 conn(0x7f18201608d0 msgr2=0x7f18200cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f181ffff700 1 --2- 172.21.15.3:0/2501441099 >> v2:172.21.15.32:3300/0 conn(0x7f18201608d0 0x7f18200cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f181ffff700 1 -- 172.21.15.3:0/2501441099 >> v1:172.21.15.3:6789/0 conn(0x7f1820160d20 legacy=0x7f18200d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f181ffff700 1 -- 172.21.15.3:0/2501441099 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18201600d0 con 0x7f1820153670 2023-03-03T06:25:47.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f5ff38e6700 1 Processor -- start 2023-03-03T06:25:47.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f181ffff700 1 --2- 172.21.15.3:0/2501441099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1820153670 0x7f1820071350 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1818009e50 tx=0x7f181800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6ec2c037acdf075 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f5d7b29f700 1 -- start start 2023-03-03T06:25:47.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f181effd700 1 -- 172.21.15.3:0/2501441099 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181800e070 con 0x7f1820153670 2023-03-03T06:25:47.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f181effd700 1 -- 172.21.15.3:0/2501441099 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181800b910 con 0x7f1820153670 2023-03-03T06:25:47.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f181effd700 1 -- 172.21.15.3:0/2501441099 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181800bc80 con 0x7f1820153670 2023-03-03T06:25:47.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.791+0000 7f5ff38e6700 1 -- start start 2023-03-03T06:25:47.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f18256a4700 1 -- 172.21.15.3:0/2501441099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1820153670 msgr2=0x7f1820071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f18256a4700 1 --2- 172.21.15.3:0/2501441099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1820153670 0x7f1820071350 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1818009e50 tx=0x7f181800b330 comp rx=0 tx=0).stop 2023-03-03T06:25:47.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f18256a4700 1 -- 172.21.15.3:0/2501441099 shutdown_connections 2023-03-03T06:25:47.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f18256a4700 1 --2- 172.21.15.3:0/2501441099 >> v2:172.21.15.32:3300/0 conn(0x7f18201608d0 0x7f18200cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f18256a4700 1 --2- 172.21.15.3:0/2501441099 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1820153670 0x7f1820071350 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f18256a4700 1 -- 172.21.15.3:0/2501441099 >> 172.21.15.3:0/2501441099 conn(0x7f182006e5c0 msgr2=0x7f182006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:47.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f18256a4700 1 -- 172.21.15.3:0/2501441099 shutdown_connections 2023-03-03T06:25:47.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f18256a4700 1 -- 172.21.15.3:0/2501441099 wait complete. 2023-03-03T06:25:47.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7b29f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d740cfb60 0x7f5d7414b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7b29f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d74153670 0x7f5d740b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7b29f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d741608d0 0x7f5d740b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7b29f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d74150210 con 0x7f5d741608d0 2023-03-03T06:25:47.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7b29f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d740b2cd0 con 0x7f5d740cfb60 2023-03-03T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7b29f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d740b2fb0 con 0x7f5d74153670 2023-03-03T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7a29d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d740cfb60 0x7f5d7414b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7aa9e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d741608d0 0x7f5d740b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7a29d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d740cfb60 0x7f5d7414b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39992/0 (socket says 172.21.15.3:39992) 2023-03-03T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7a29d700 1 -- 172.21.15.3:0/2135546716 learned_addr learned my addr 172.21.15.3:0/2135546716 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d79a9c700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d74153670 0x7f5d740b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5ff38e6700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fec0b8380 0x7f5fec0d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5ff38e6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fec160d70 0x7f5fec160160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5ff38e6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fec0d3c20 con 0x7f5fec0b8750 2023-03-03T06:25:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5ff38e6700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fec0cd400 con 0x7f5fec0b8380 2023-03-03T06:25:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5ff38e6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0cdbd0 con 0x7f5fec160d70 2023-03-03T06:25:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7a29d700 1 -- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d74153670 msgr2=0x7f5d740b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7a29d700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d74153670 0x7f5d740b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7a29d700 1 -- 172.21.15.3:0/2135546716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d741608d0 msgr2=0x7f5d740b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7a29d700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d741608d0 0x7f5d740b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7a29d700 1 -- 172.21.15.3:0/2135546716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d64009570 con 0x7f5d740cfb60 2023-03-03T06:25:47.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7aa9e700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d741608d0 0x7f5d740b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:47.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5ff1682700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fec0b8380 0x7f5fec0d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5ff1682700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fec0b8380 0x7f5fec0d36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39996/0 (socket says 172.21.15.3:39996) 2023-03-03T06:25:47.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7a29d700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d740cfb60 0x7f5d7414b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00cc20 tx=0x7f5d6c00cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5ff1682700 1 -- 172.21.15.3:0/3693685563 learned_addr learned my addr 172.21.15.3:0/3693685563 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:47.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d6affd700 1 -- 172.21.15.3:0/2135546716 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c003c20 con 0x7f5d740cfb60 2023-03-03T06:25:47.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d6affd700 1 -- 172.21.15.3:0/2135546716 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6c003d80 con 0x7f5d740cfb60 2023-03-03T06:25:47.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d6affd700 1 -- 172.21.15.3:0/2135546716 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c00aae0 con 0x7f5d740cfb60 2023-03-03T06:25:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.795+0000 7f5d7b29f700 1 -- 172.21.15.3:0/2135546716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d740cd9a0 con 0x7f5d740cfb60 2023-03-03T06:25:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f5ff1682700 1 -- 172.21.15.3:0/3693685563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fec160d70 msgr2=0x7f5fec160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f5ff1682700 1 --2- 172.21.15.3:0/3693685563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fec160d70 0x7f5fec160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f5ff1682700 1 -- 172.21.15.3:0/3693685563 >> v1:172.21.15.32:6789/0 conn(0x7f5fec0b8750 legacy=0x7f5fec0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f5ff1682700 1 -- 172.21.15.3:0/3693685563 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec0d0190 con 0x7f5fec0b8380 2023-03-03T06:25:47.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f18256a4700 1 Processor -- start 2023-03-03T06:25:47.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f18256a4700 1 -- start start 2023-03-03T06:25:47.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f5d7b29f700 1 -- 172.21.15.3:0/2135546716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d741600d0 con 0x7f5d740cfb60 2023-03-03T06:25:47.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f5ff1682700 1 --2- 172.21.15.3:0/3693685563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fec0b8380 0x7f5fec0d36e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009910 tx=0x7f5fe8009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=910ca39d6563445b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f18256a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1820153670 0x7f18200c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f18256a4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18201608d0 0x7f182014e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f5fe3fff700 1 -- 172.21.15.3:0/3693685563 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8004230 con 0x7f5fec0b8380 2023-03-03T06:25:47.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f5fe3fff700 1 -- 172.21.15.3:0/3693685563 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8004390 con 0x7f5fec0b8380 2023-03-03T06:25:47.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f5fe3fff700 1 -- 172.21.15.3:0/3693685563 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8009e60 con 0x7f5fec0b8380 2023-03-03T06:25:47.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3693685563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fec0b8380 msgr2=0x7f5fec0d36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f5ff38e6700 1 --2- 172.21.15.3:0/3693685563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fec0b8380 0x7f5fec0d36e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009910 tx=0x7f5fe8009c20 comp rx=0 tx=0).stop 2023-03-03T06:25:47.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f181f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18201608d0 0x7f182014e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f181f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18201608d0 0x7f182014e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43646/0 (socket says 172.21.15.3:43646) 2023-03-03T06:25:47.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f181f7fe700 1 -- 172.21.15.3:0/3209476899 learned_addr learned my addr 172.21.15.3:0/3209476899 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:47.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.799+0000 7f18256a4700 1 --2- 172.21.15.3:0/3209476899 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1820160d20 0x7f1820152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3693685563 shutdown_connections 2023-03-03T06:25:47.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f5ff38e6700 1 --2- 172.21.15.3:0/3693685563 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fec160d70 0x7f5fec160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f5ff38e6700 1 --2- 172.21.15.3:0/3693685563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fec0b8380 0x7f5fec0d36e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3693685563 >> 172.21.15.3:0/3693685563 conn(0x7f5fec06e5c0 msgr2=0x7f5fec06e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:47.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f18256a4700 1 -- 172.21.15.3:0/3209476899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1820151410 con 0x7f18201608d0 2023-03-03T06:25:47.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f18256a4700 1 -- 172.21.15.3:0/3209476899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18200c9360 con 0x7f1820153670 2023-03-03T06:25:47.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f18256a4700 1 -- 172.21.15.3:0/3209476899 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182015e790 con 0x7f1820160d20 2023-03-03T06:25:47.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f181f7fe700 1 -- 172.21.15.3:0/3209476899 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1820160d20 msgr2=0x7f1820152400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-03-03T06:25:47.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f181f7fe700 1 --2- 172.21.15.3:0/3209476899 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1820160d20 0x7f1820152400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f181f7fe700 1 -- 172.21.15.3:0/3209476899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1820153670 msgr2=0x7f18200c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f181f7fe700 1 --2- 172.21.15.3:0/3209476899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1820153670 0x7f18200c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f181f7fe700 1 -- 172.21.15.3:0/3209476899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1818009b30 con 0x7f18201608d0 2023-03-03T06:25:47.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f5d6affd700 1 -- 172.21.15.3:0/2135546716 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5d6c010050 con 0x7f5d740cfb60 2023-03-03T06:25:47.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f181f7fe700 1 --2- 172.21.15.3:0/3209476899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18201608d0 0x7f182014e480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f181400e9d0 tx=0x7f181400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3693685563 shutdown_connections 2023-03-03T06:25:47.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f5d6affd700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5d58064f30 0x7f5d580673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f5d6affd700 1 -- 172.21.15.3:0/2135546716 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5d6c082080 con 0x7f5d740cfb60 2023-03-03T06:25:47.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f609b484700 1 Processor -- start 2023-03-03T06:25:47.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3693685563 wait complete. 2023-03-03T06:25:47.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f181cff9700 1 -- 172.21.15.3:0/3209476899 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181400c850 con 0x7f18201608d0 2023-03-03T06:25:47.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f181cff9700 1 -- 172.21.15.3:0/3209476899 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181400c9b0 con 0x7f18201608d0 2023-03-03T06:25:47.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f181cff9700 1 -- 172.21.15.3:0/3209476899 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181400cd20 con 0x7f18201608d0 2023-03-03T06:25:47.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f18256a4700 1 -- 172.21.15.3:0/3209476899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18200cd9a0 con 0x7f18201608d0 2023-03-03T06:25:47.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f5d7b29f700 1 -- 172.21.15.3:0/2135546716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d740cdb30 con 0x7f5d740cfb60 2023-03-03T06:25:47.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f5d79a9c700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5d58064f30 0x7f5d580673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.803+0000 7f609b484700 1 -- start start 2023-03-03T06:25:47.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f5ff38e6700 1 Processor -- start 2023-03-03T06:25:47.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7fa3b09f2700 1 Processor -- start 2023-03-03T06:25:47.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f5d79a9c700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5d58064f30 0x7f5d580673e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d64000c00 tx=0x7f5d64008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f609b484700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f608c0e04b0 0x7f608c039bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f609b484700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f608c00e420 0x7f608c0387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f609b484700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c051930 con 0x7f608c056860 2023-03-03T06:25:47.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f609b484700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c0513b0 con 0x7f608c0e04b0 2023-03-03T06:25:47.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f609b484700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0394b0 con 0x7f608c00e420 2023-03-03T06:25:47.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f609ac83700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f608c056860 0x7f608c0554f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33760/0 (socket says 172.21.15.3:33760) 2023-03-03T06:25:47.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f609ac83700 1 -- 172.21.15.3:0/2773143473 learned_addr learned my addr 172.21.15.3:0/2773143473 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:47.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f5ff38e6700 1 -- start start 2023-03-03T06:25:47.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f5ff38e6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fec0b8380 0x7f5fec15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f5ff38e6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fec160d70 0x7f5fec0c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f5ff38e6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fec0c8460 0x7f5fec0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7fa3b09f2700 1 -- start start 2023-03-03T06:25:47.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7fa3b09f2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3a40e0560 0x7fa3a40e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7fa3b09f2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3a40e1610 0x7fa3a404f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f5ff38e6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec15b6c0 con 0x7f5fec160d70 2023-03-03T06:25:47.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7fa3b09f2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a4036c50 con 0x7fa3a404faa0 2023-03-03T06:25:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7fa3b09f2700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a40e50c0 con 0x7fa3a40e0560 2023-03-03T06:25:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7fa3b09f2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a40e4820 con 0x7fa3a40e1610 2023-03-03T06:25:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f5ff1682700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fec0b8380 0x7f5fec15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f5ff1682700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fec0b8380 0x7f5fec15b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:55558/0 (socket says 172.21.15.3:55558) 2023-03-03T06:25:47.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f5ff1682700 1 -- 172.21.15.3:0/3925755914 learned_addr learned my addr 172.21.15.3:0/3925755914 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:47.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f5ff1e83700 1 --2- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fec0c8460 0x7f5fec0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578958003 0 0) 0x7f608c051930 con 0x7f608c056860 2023-03-03T06:25:47.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60800035d0 con 0x7f608c056860 2023-03-03T06:25:47.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3925755914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec15d7b0 con 0x7f5fec0c8460 2023-03-03T06:25:47.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3925755914 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0b65f0 con 0x7f5fec0b8380 2023-03-03T06:25:47.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7fa3aad9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa3a404faa0 0x7fa3a40370e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:56618/0 (socket says 172.21.15.3:56618) 2023-03-03T06:25:47.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7fa3aad9d700 1 -- 172.21.15.3:0/2797169552 learned_addr learned my addr 172.21.15.3:0/2797169552 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:47.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f609a482700 1 --2- 172.21.15.3:0/2773143473 >> v2:172.21.15.32:3300/0 conn(0x7f608c0e04b0 0x7f608c039bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099c81700 1 --2- 172.21.15.3:0/2773143473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f608c00e420 0x7f608c0387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7fa3a9d9b700 1 --2- 172.21.15.3:0/2797169552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3a40e1610 0x7fa3a404f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.807+0000 7f181cff9700 1 -- 172.21.15.3:0/3209476899 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1814019030 con 0x7f18201608d0 2023-03-03T06:25:47.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f181cff9700 1 --2- 172.21.15.3:0/3209476899 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f180c064f60 0x7f180c067410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f5ff1e83700 1 -- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fec0b8380 msgr2=0x7f5fec15b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f5ff1e83700 1 --2- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fec0b8380 0x7f5fec15b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f5ff1e83700 1 -- 172.21.15.3:0/3925755914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fec160d70 msgr2=0x7f5fec0c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f5ff1e83700 1 --2- 172.21.15.3:0/3925755914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fec160d70 0x7f5fec0c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f5ff1e83700 1 -- 172.21.15.3:0/3925755914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe8009570 con 0x7f5fec0c8460 2023-03-03T06:25:47.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4171434463 0 0) 0x7f60800035d0 con 0x7f608c056860 2023-03-03T06:25:47.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f18256a4700 1 -- 172.21.15.3:0/3209476899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1820071890 con 0x7f18201608d0 2023-03-03T06:25:47.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f5ff1e83700 1 --2- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fec0c8460 0x7f5fec0c8e60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400be20 tx=0x7f5fe4009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f608c00e420 msgr2=0x7f608c0387e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:47.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 --2- 172.21.15.3:0/2773143473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f608c00e420 0x7f608c0387e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 >> v2:172.21.15.32:3300/0 conn(0x7f608c0e04b0 msgr2=0x7f608c039bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:47.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 --2- 172.21.15.3:0/2773143473 >> v2:172.21.15.32:3300/0 conn(0x7f608c0e04b0 0x7f608c039bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7fa3a9d9b700 1 -- 172.21.15.3:0/2797169552 >> v2:172.21.15.3:3300/0 conn(0x7fa3a40e0560 msgr2=0x7fa3a40e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:47.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7fa3a9d9b700 1 --2- 172.21.15.3:0/2797169552 >> v2:172.21.15.3:3300/0 conn(0x7fa3a40e0560 0x7fa3a40e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7fa3a9d9b700 1 -- 172.21.15.3:0/2797169552 >> v1:172.21.15.32:6789/0 conn(0x7fa3a404faa0 legacy=0x7fa3a40370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7fa3a9d9b700 1 -- 172.21.15.3:0/2797169552 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a40e4b00 con 0x7fa3a40e1610 2023-03-03T06:25:47.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c0515c0 con 0x7f608c056860 2023-03-03T06:25:47.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f6090001670 con 0x7f608c056860 2023-03-03T06:25:47.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 >> v1:172.21.15.3:6789/0 conn(0x7f608c056860 legacy=0x7f608c0554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 --2- 172.21.15.3:0/2773143473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6080003a20 0x7f6080003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 --2- 172.21.15.3:0/2773143473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6080004400 0x7f60800048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 --2- 172.21.15.3:0/2773143473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6080005020 0x7f608000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0513b0 con 0x7f6080005020 2023-03-03T06:25:47.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0394b0 con 0x7f6080004400 2023-03-03T06:25:47.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608000bfa0 con 0x7f6080003a20 2023-03-03T06:25:47.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f181ffff700 1 --2- 172.21.15.3:0/3209476899 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f180c064f60 0x7f180c067410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7fa3a9d9b700 1 --2- 172.21.15.3:0/2797169552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3a40e1610 0x7fa3a404f560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009850 tx=0x7fa3a0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29acbffd722ee5d3 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f609ac83700 1 --2- 172.21.15.3:0/2773143473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6080005020 0x7f608000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7fa3a959a700 1 -- 172.21.15.3:0/2797169552 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0011830 con 0x7fa3a40e1610 2023-03-03T06:25:47.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7fa3a959a700 1 -- 172.21.15.3:0/2797169552 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a0011990 con 0x7fa3a40e1610 2023-03-03T06:25:47.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7fa3a959a700 1 -- 172.21.15.3:0/2797169552 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0011d20 con 0x7fa3a40e1610 2023-03-03T06:25:47.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f6099c81700 1 --2- 172.21.15.3:0/2773143473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6080004400 0x7f60800048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.811+0000 7f18256a4700 1 -- 172.21.15.3:0/3209476899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18200cdb30 con 0x7f18201608d0 2023-03-03T06:25:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f609a482700 1 --2- 172.21.15.3:0/2773143473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6080003a20 0x7f6080003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f181ffff700 1 --2- 172.21.15.3:0/3209476899 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f180c064f60 0x7f180c067410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f18200c6690 tx=0x7f1818010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f181cff9700 1 -- 172.21.15.3:0/3209476899 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f181405b520 con 0x7f18201608d0 2023-03-03T06:25:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f609ac83700 1 -- 172.21.15.3:0/2773143473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6080003a20 msgr2=0x7f6080003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f609ac83700 1 --2- 172.21.15.3:0/2773143473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6080003a20 0x7f6080003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f609ac83700 1 -- 172.21.15.3:0/2773143473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6080004400 msgr2=0x7f60800048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f609ac83700 1 --2- 172.21.15.3:0/2773143473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6080004400 0x7f60800048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f609ac83700 1 -- 172.21.15.3:0/2773143473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c0515c0 con 0x7f6080005020 2023-03-03T06:25:47.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7fa3b09f2700 1 -- 172.21.15.3:0/2797169552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3a40e1610 msgr2=0x7fa3a404f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7fa3b09f2700 1 --2- 172.21.15.3:0/2797169552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3a40e1610 0x7fa3a404f560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009850 tx=0x7fa3a0009b60 comp rx=0 tx=0).stop 2023-03-03T06:25:47.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f609a482700 1 --2- 172.21.15.3:0/2773143473 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6080003a20 0x7f6080003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:25:47.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f609ac83700 1 --2- 172.21.15.3:0/2773143473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6080005020 0x7f608000ba30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f609000c2c0 tx=0x7f60800058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf06386766ae647 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7fa3b09f2700 1 -- 172.21.15.3:0/2797169552 shutdown_connections 2023-03-03T06:25:47.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7fa3b09f2700 1 --2- 172.21.15.3:0/2797169552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3a40e1610 0x7fa3a404f560 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7fa3b09f2700 1 --2- 172.21.15.3:0/2797169552 >> v2:172.21.15.3:3300/0 conn(0x7fa3a40e0560 0x7fa3a40e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7fa3b09f2700 1 -- 172.21.15.3:0/2797169552 >> 172.21.15.3:0/2797169552 conn(0x7fa3a4051a20 msgr2=0x7fa3a4051e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7fa3b09f2700 1 -- 172.21.15.3:0/2797169552 shutdown_connections 2023-03-03T06:25:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7fa3b09f2700 1 -- 172.21.15.3:0/2797169552 wait complete. 2023-03-03T06:25:47.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f5d6affd700 1 -- 172.21.15.3:0/2135546716 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5d6c04fec0 con 0x7f5d740cfb60 2023-03-03T06:25:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f5fe1ffb700 1 -- 172.21.15.3:0/3925755914 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe400cc10 con 0x7f5fec0c8460 2023-03-03T06:25:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f5fe1ffb700 1 -- 172.21.15.3:0/3925755914 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe400cd90 con 0x7f5fec0c8460 2023-03-03T06:25:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3925755914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fec0cd610 con 0x7f5fec0c8460 2023-03-03T06:25:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.815+0000 7f5fe1ffb700 1 -- 172.21.15.3:0/3925755914 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4014680 con 0x7f5fec0c8460 2023-03-03T06:25:47.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3925755914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec0cf000 con 0x7f5fec0c8460 2023-03-03T06:25:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7fa3b09f2700 1 Processor -- start 2023-03-03T06:25:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7fa3b09f2700 1 -- start start 2023-03-03T06:25:47.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7f47690d8700 1 Processor -- start 2023-03-03T06:25:47.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3925755914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fec0cd8f0 con 0x7f5fec0c8460 2023-03-03T06:25:47.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7fa3b09f2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a404faa0 0x7fa3a404a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7fa3b09f2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3a40e0560 0x7fa3a40d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7fa3b09f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3a40e1610 0x7fa3a40ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7fa3b09f2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a404a700 con 0x7fa3a404faa0 2023-03-03T06:25:47.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7fa3b09f2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a404dfc0 con 0x7fa3a40e1610 2023-03-03T06:25:47.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7fa3b09f2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a4048aa0 con 0x7fa3a40e0560 2023-03-03T06:25:47.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7fa3a9d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3a40e0560 0x7fa3a40d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7fa3a9d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3a40e0560 0x7fa3a40d25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:55586/0 (socket says 172.21.15.3:55586) 2023-03-03T06:25:47.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7fa3a9d9b700 1 -- 172.21.15.3:0/1182566552 learned_addr learned my addr 172.21.15.3:0/1182566552 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:47.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7fa3aad9d700 1 --2- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3a40e1610 0x7fa3a40ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.819+0000 7f47690d8700 1 -- start start 2023-03-03T06:25:47.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7fa3a9d9b700 1 -- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3a40e1610 msgr2=0x7fa3a40ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7fa3a9d9b700 1 --2- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3a40e1610 0x7fa3a40ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7fa3a9d9b700 1 -- 172.21.15.3:0/1182566552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a404faa0 msgr2=0x7fa3a404a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7fa3a9d9b700 1 --2- 172.21.15.3:0/1182566552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a404faa0 0x7fa3a404a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7fa3a9d9b700 1 -- 172.21.15.3:0/1182566552 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a00094b0 con 0x7fa3a40e0560 2023-03-03T06:25:47.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f47690d8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f475c0520e0 0x7f475c0de880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f47690d8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f475c053c40 con 0x7f475c036a20 2023-03-03T06:25:47.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f47690d8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f475c0e31e0 con 0x7f475c0365d0 2023-03-03T06:25:47.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f47690d8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475c006860 con 0x7f475c0520e0 2023-03-03T06:25:47.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7fa3a9d9b700 1 --2- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3a40e0560 0x7fa3a40d25c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009fd0 tx=0x7fa3a0004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f476359e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f475c036a20 0x7f475c0e2ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33770/0 (socket says 172.21.15.3:33770) 2023-03-03T06:25:47.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f4762d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f475c0520e0 0x7f475c0de880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f476359e700 1 -- 172.21.15.3:0/619640574 learned_addr learned my addr 172.21.15.3:0/619640574 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:47.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f4762d9d700 1 -- 172.21.15.3:0/619640574 >> v1:172.21.15.32:6789/0 conn(0x7f475c0365d0 legacy=0x7f475c0e23c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-03T06:25:47.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f4762d9d700 1 -- 172.21.15.3:0/619640574 >> v1:172.21.15.3:6789/0 conn(0x7f475c036a20 legacy=0x7f475c0e2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7fa39affd700 1 -- 172.21.15.3:0/1182566552 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00045d0 con 0x7fa3a40e0560 2023-03-03T06:25:47.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7fa39affd700 1 -- 172.21.15.3:0/1182566552 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a001f070 con 0x7fa3a40e0560 2023-03-03T06:25:47.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7fa39affd700 1 -- 172.21.15.3:0/1182566552 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0003b10 con 0x7fa3a40e0560 2023-03-03T06:25:47.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f4762d9d700 1 -- 172.21.15.3:0/619640574 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c005f00 con 0x7f475c0520e0 2023-03-03T06:25:47.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7fa3b09f2700 1 -- 172.21.15.3:0/1182566552 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a40e1a80 con 0x7fa3a40e0560 2023-03-03T06:25:47.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f4762d9d700 1 --2- 172.21.15.3:0/619640574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f475c0520e0 0x7f475c0de880 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f476404f390 tx=0x7f47640617d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5484efce78b2f2 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7fa3b09f2700 1 -- 172.21.15.3:0/1182566552 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a40e4de0 con 0x7fa3a40e0560 2023-03-03T06:25:47.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f5fe1ffb700 1 -- 172.21.15.3:0/3925755914 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5fe401b030 con 0x7f5fec0c8460 2023-03-03T06:25:47.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f5fe1ffb700 1 --2- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5fd0064f60 0x7f5fd0067410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.823+0000 7f5fe1ffb700 1 -- 172.21.15.3:0/3925755914 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5fe4082020 con 0x7f5fec0c8460 2023-03-03T06:25:47.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7f5ff1682700 1 --2- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5fd0064f60 0x7f5fd0067410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7fa3b09f2700 1 -- 172.21.15.3:0/1182566552 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a40e0930 con 0x7fa3a40e0560 2023-03-03T06:25:47.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7f181cff9700 1 -- 172.21.15.3:0/3209476899 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f181404a400 con 0x7f18201608d0 2023-03-03T06:25:47.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7f5ff1682700 1 --2- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5fd0064f60 0x7f5fd0067410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009c60 tx=0x7f5fe8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090007c80 con 0x7f6080005020 2023-03-03T06:25:47.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609000ce20 con 0x7f6080005020 2023-03-03T06:25:47.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7f6099480700 1 -- 172.21.15.3:0/2773143473 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090005ba0 con 0x7f6080005020 2023-03-03T06:25:47.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7f4761d9b700 1 -- 172.21.15.3:0/619640574 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764066cc0 con 0x7f475c0520e0 2023-03-03T06:25:47.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7f4761d9b700 1 -- 172.21.15.3:0/619640574 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4764066e40 con 0x7f475c0520e0 2023-03-03T06:25:47.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7f4761d9b700 1 -- 172.21.15.3:0/619640574 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640720c0 con 0x7f475c0520e0 2023-03-03T06:25:47.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7f609b484700 1 -- 172.21.15.3:0/2773143473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6080005020 msgr2=0x7f608000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7f609b484700 1 --2- 172.21.15.3:0/2773143473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6080005020 0x7f608000ba30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f609000c2c0 tx=0x7f60800058a0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7f609ac83700 1 -- 172.21.15.3:0/2773143473 reap_dead start 2023-03-03T06:25:47.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7f47690d8700 1 -- 172.21.15.3:0/619640574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f475c0520e0 msgr2=0x7f475c0de880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7f47690d8700 1 --2- 172.21.15.3:0/619640574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f475c0520e0 0x7f475c0de880 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f476404f390 tx=0x7f47640617d0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7f609b484700 1 -- 172.21.15.3:0/2773143473 shutdown_connections 2023-03-03T06:25:47.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7f609b484700 1 -- 172.21.15.3:0/2773143473 >> 172.21.15.3:0/2773143473 conn(0x7f608c00c250 msgr2=0x7f608c00c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:47.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7f5fe1ffb700 1 -- 172.21.15.3:0/3925755914 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5fe404fde0 con 0x7f5fec0c8460 2023-03-03T06:25:47.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7f609b484700 1 -- 172.21.15.3:0/2773143473 shutdown_connections 2023-03-03T06:25:47.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.827+0000 7fa39affd700 1 -- 172.21.15.3:0/1182566552 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa3a000f050 con 0x7fa3a40e0560 2023-03-03T06:25:47.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7fa39affd700 1 --2- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa38c064fb0 0x7fa38c067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7fa39affd700 1 -- 172.21.15.3:0/1182566552 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa3a0021070 con 0x7fa3a40e0560 2023-03-03T06:25:47.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7f47690d8700 1 -- 172.21.15.3:0/619640574 shutdown_connections 2023-03-03T06:25:47.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7f47690d8700 1 --2- 172.21.15.3:0/619640574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f475c0520e0 0x7f475c0de880 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7f47690d8700 1 -- 172.21.15.3:0/619640574 >> 172.21.15.3:0/619640574 conn(0x7f475c0515d0 msgr2=0x7f475c0519d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:47.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7f609b484700 1 -- 172.21.15.3:0/2773143473 wait complete. 2023-03-03T06:25:47.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7fa3aa59c700 1 --2- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa38c064fb0 0x7fa38c067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.831+0000 7fa3aa59c700 1 --2- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa38c064fb0 0x7fa38c067460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009730 tx=0x7fa39c006ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609b484700 1 Processor -- start 2023-03-03T06:25:47.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609b484700 1 -- start start 2023-03-03T06:25:47.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f47690d8700 1 -- 172.21.15.3:0/619640574 shutdown_connections 2023-03-03T06:25:47.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609b484700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6080005020 0x7f608c04eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609b484700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f608c04b4a0 0x7f608c03c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609b484700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f608c17c5a0 0x7f608c0d7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609b484700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c050200 con 0x7f6080005020 2023-03-03T06:25:47.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609b484700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c04f5f0 con 0x7f608c17c5a0 2023-03-03T06:25:47.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609b484700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c04f8d0 con 0x7f608c04b4a0 2023-03-03T06:25:47.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609a482700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6080005020 0x7f608c04eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609a482700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6080005020 0x7f608c04eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43710/0 (socket says 172.21.15.3:43710) 2023-03-03T06:25:47.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609a482700 1 -- 172.21.15.3:0/161358904 learned_addr learned my addr 172.21.15.3:0/161358904 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:47.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f6099c81700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f608c04b4a0 0x7f608c03c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609ac83700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f608c17c5a0 0x7f608c0d7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609a482700 1 -- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f608c04b4a0 msgr2=0x7f608c03c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609a482700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f608c04b4a0 0x7f608c03c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609a482700 1 -- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f608c17c5a0 msgr2=0x7f608c0d7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609a482700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f608c17c5a0 0x7f608c0d7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f609a482700 1 -- 172.21.15.3:0/161358904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c0e0880 con 0x7f6080005020 2023-03-03T06:25:47.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7fa39affd700 1 -- 172.21.15.3:0/1182566552 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa3a0051920 con 0x7fa3a40e0560 2023-03-03T06:25:47.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.835+0000 7f47690d8700 1 -- 172.21.15.3:0/619640574 wait complete. 2023-03-03T06:25:47.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.839+0000 7f609a482700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6080005020 0x7f608c04eef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f60840109f0 tx=0x7f6084010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.839+0000 7f608affd700 1 -- 172.21.15.3:0/161358904 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084004220 con 0x7f6080005020 2023-03-03T06:25:47.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.839+0000 7f608affd700 1 -- 172.21.15.3:0/161358904 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6084004380 con 0x7f6080005020 2023-03-03T06:25:47.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.839+0000 7f608affd700 1 -- 172.21.15.3:0/161358904 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084010e90 con 0x7f6080005020 2023-03-03T06:25:47.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.839+0000 7f47690d8700 1 Processor -- start 2023-03-03T06:25:47.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.839+0000 7f609b484700 1 -- 172.21.15.3:0/161358904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c03a0f0 con 0x7f6080005020 2023-03-03T06:25:47.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.839+0000 7f609b484700 1 -- 172.21.15.3:0/161358904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c056240 con 0x7f6080005020 2023-03-03T06:25:47.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.839+0000 7f609b484700 1 -- 172.21.15.3:0/161358904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607c005310 con 0x7f6080005020 2023-03-03T06:25:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f47690d8700 1 -- start start 2023-03-03T06:25:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f47690d8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f475c0365d0 0x7f475c04e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f47690d8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c036a20 0x7f475c0dd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f47690d8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c0520e0 0x7f475c184560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f47690d8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c0ddef0 con 0x7f475c0365d0 2023-03-03T06:25:47.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f47690d8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c04ec70 con 0x7f475c0520e0 2023-03-03T06:25:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f47690d8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475c0dd790 con 0x7f475c036a20 2023-03-03T06:25:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f4762d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f475c0365d0 0x7f475c04e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f4762d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f475c0365d0 0x7f475c04e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43712/0 (socket says 172.21.15.3:43712) 2023-03-03T06:25:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f4762d9d700 1 -- 172.21.15.3:0/2112932403 learned_addr learned my addr 172.21.15.3:0/2112932403 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f476259c700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c036a20 0x7f475c0dd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f476359e700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c0520e0 0x7f475c184560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f4762d9d700 1 -- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c036a20 msgr2=0x7f475c0dd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f4762d9d700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c036a20 0x7f475c0dd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f4762d9d700 1 -- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c0520e0 msgr2=0x7f475c184560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f4762d9d700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c0520e0 0x7f475c184560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f4762d9d700 1 -- 172.21.15.3:0/2112932403 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4764066950 con 0x7f475c0365d0 2023-03-03T06:25:47.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.843+0000 7f608affd700 1 -- 172.21.15.3:0/161358904 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6084023070 con 0x7f6080005020 2023-03-03T06:25:47.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.847+0000 7f608affd700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6078064fb0 0x7f6078067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.847+0000 7f608affd700 1 -- 172.21.15.3:0/161358904 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6084082820 con 0x7f6080005020 2023-03-03T06:25:47.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.847+0000 7f6099c81700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6078064fb0 0x7f6078067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.847+0000 7f6099c81700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6078064fb0 0x7f6078067460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6094068530 tx=0x7f6094072040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.847+0000 7f4762d9d700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f475c0365d0 0x7f475c04e5b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4764063970 tx=0x7f476404e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.851+0000 7f474b7fe700 1 -- 172.21.15.3:0/2112932403 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764061510 con 0x7f475c0365d0 2023-03-03T06:25:47.871 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.851+0000 7f608affd700 1 -- 172.21.15.3:0/161358904 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f60840505d0 con 0x7f6080005020 2023-03-03T06:25:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.851+0000 7f474b7fe700 1 -- 172.21.15.3:0/2112932403 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4764096040 con 0x7f475c0365d0 2023-03-03T06:25:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.851+0000 7f47690d8700 1 -- 172.21.15.3:0/2112932403 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c006480 con 0x7f475c0365d0 2023-03-03T06:25:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.851+0000 7f47690d8700 1 -- 172.21.15.3:0/2112932403 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475c0061a0 con 0x7f475c0365d0 2023-03-03T06:25:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.851+0000 7f474b7fe700 1 -- 172.21.15.3:0/2112932403 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640609c0 con 0x7f475c0365d0 2023-03-03T06:25:47.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.855+0000 7f474b7fe700 1 -- 172.21.15.3:0/2112932403 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4764060be0 con 0x7f475c0365d0 2023-03-03T06:25:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.855+0000 7f474b7fe700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4750064fb0 0x7f4750067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.855+0000 7f474b7fe700 1 -- 172.21.15.3:0/2112932403 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f47640fc040 con 0x7f475c0365d0 2023-03-03T06:25:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.859+0000 7f476259c700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4750064fb0 0x7f4750067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.859+0000 7f47690d8700 1 -- 172.21.15.3:0/2112932403 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4744005310 con 0x7f475c0365d0 2023-03-03T06:25:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.867+0000 7f474b7fe700 1 -- 172.21.15.3:0/2112932403 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4764101060 con 0x7f475c0365d0 2023-03-03T06:25:47.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:47.867+0000 7f476259c700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4750064fb0 0x7f4750067460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f475c04ee50 tx=0x7f4758008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:48.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.095+0000 7f7d776a7700 1 -- 172.21.15.3:0/2888984364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7d7015dcb0 con 0x7f7d700b1b90 2023-03-03T06:25:48.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.095+0000 7f7d65ffb700 1 -- 172.21.15.3:0/2888984364 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d7015dcb0 con 0x7f7d700b1b90 2023-03-03T06:25:48.103 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2023-03-03T06:25:48.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.099+0000 7f1fb1a5b700 1 -- 172.21.15.3:0/122648050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1fac0b1fd0 con 0x7f1fac0c9890 2023-03-03T06:25:48.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.099+0000 7f1f93fff700 1 -- 172.21.15.3:0/122648050 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f9c053380 con 0x7f1fac0c9890 2023-03-03T06:25:48.105 INFO:teuthology.orchestra.run.smithi003.stdout:55834574861 2023-03-03T06:25:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.103+0000 7f7d537fe700 1 -- 172.21.15.3:0/2888984364 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d54064f30 msgr2=0x7f7d540673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.103+0000 7f7d537fe700 1 --2- 172.21.15.3:0/2888984364 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d54064f30 0x7f7d540673e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000fb0 tx=0x7f7d6c001b70 comp rx=0 tx=0).stop 2023-03-03T06:25:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.103+0000 7f7d537fe700 1 -- 172.21.15.3:0/2888984364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d700b1b90 msgr2=0x7f7d700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.103+0000 7f7d537fe700 1 --2- 172.21.15.3:0/2888984364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d700b1b90 0x7f7d700b2000 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7d60004000 tx=0x7f7d60001120 comp rx=0 tx=0).stop 2023-03-03T06:25:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.103+0000 7f7d537fe700 1 -- 172.21.15.3:0/2888984364 shutdown_connections 2023-03-03T06:25:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.103+0000 7f7d537fe700 1 --2- 172.21.15.3:0/2888984364 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d54064f30 0x7f7d540673e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.103+0000 7f7d537fe700 1 --2- 172.21.15.3:0/2888984364 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7d7015a6f0 0x7f7d7015ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.103+0000 7f7d537fe700 1 --2- 172.21.15.3:0/2888984364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d700b1b90 0x7f7d700b2000 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.103+0000 7f7d537fe700 1 --2- 172.21.15.3:0/2888984364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d5c004450 0x7f7d701f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.103+0000 7f7d537fe700 1 -- 172.21.15.3:0/2888984364 >> 172.21.15.3:0/2888984364 conn(0x7f7d70060be0 msgr2=0x7f7d700cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:48.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f7d537fe700 1 -- 172.21.15.3:0/2888984364 shutdown_connections 2023-03-03T06:25:48.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f7d537fe700 1 -- 172.21.15.3:0/2888984364 wait complete. 2023-03-03T06:25:48.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f1f927fc700 1 -- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1f98064fb0 msgr2=0x7f1f98067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f1f927fc700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1f98064fb0 0x7f1f98067460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009ba0 tx=0x7f1fa0008030 comp rx=0 tx=0).stop 2023-03-03T06:25:48.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f1f927fc700 1 -- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fac0c9890 msgr2=0x7f1fac0c22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f1f927fc700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fac0c9890 0x7f1fac0c22d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00b3c0 tx=0x7f1f9c00b6d0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f1f927fc700 1 -- 172.21.15.3:0/122648050 shutdown_connections 2023-03-03T06:25:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f1f927fc700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1f98064fb0 0x7f1f98067460 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f1f927fc700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1fac15fa00 0x7f1fac0c2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f1f927fc700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fac0c9890 0x7f1fac0c22d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f1f927fc700 1 --2- 172.21.15.3:0/122648050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1fac0add80 0x7f1fac0c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f1f927fc700 1 -- 172.21.15.3:0/122648050 >> 172.21.15.3:0/122648050 conn(0x7f1fac060be0 msgr2=0x7f1fac0b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:48.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f1f927fc700 1 -- 172.21.15.3:0/122648050 shutdown_connections 2023-03-03T06:25:48.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.107+0000 7f1f927fc700 1 -- 172.21.15.3:0/122648050 wait complete. 2023-03-03T06:25:48.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2023-03-03T06:25:48.155 DEBUG:teuthology.parallel:result is None 2023-03-03T06:25:48.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574861 for osd.1 2023-03-03T06:25:48.156 DEBUG:teuthology.parallel:result is None 2023-03-03T06:25:48.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.159+0000 7f5d7b29f700 1 -- 172.21.15.3:0/2135546716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f5d740b30f0 con 0x7f5d740cfb60 2023-03-03T06:25:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.159+0000 7f18256a4700 1 -- 172.21.15.3:0/3209476899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f18200c9570 con 0x7f18201608d0 2023-03-03T06:25:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.159+0000 7f181cff9700 1 -- 172.21.15.3:0/3209476899 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1814017020 con 0x7f18201608d0 2023-03-03T06:25:48.166 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2023-03-03T06:25:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f5d6affd700 1 -- 172.21.15.3:0/2135546716 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5d740b30f0 con 0x7f5d740cfb60 2023-03-03T06:25:48.166 INFO:teuthology.orchestra.run.smithi003.stdout:55834574860 2023-03-03T06:25:48.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3925755914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5fec04e180 con 0x7f5fec0c8460 2023-03-03T06:25:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f18256a4700 1 -- 172.21.15.3:0/3209476899 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f180c064f60 msgr2=0x7f180c067410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f18256a4700 1 --2- 172.21.15.3:0/3209476899 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f180c064f60 0x7f180c067410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f18200c6690 tx=0x7f1818010000 comp rx=0 tx=0).stop 2023-03-03T06:25:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f18256a4700 1 -- 172.21.15.3:0/3209476899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18201608d0 msgr2=0x7f182014e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f18256a4700 1 --2- 172.21.15.3:0/3209476899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18201608d0 0x7f182014e480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f181400e9d0 tx=0x7f181400ed90 comp rx=0 tx=0).stop 2023-03-03T06:25:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f18256a4700 1 -- 172.21.15.3:0/3209476899 shutdown_connections 2023-03-03T06:25:48.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f18256a4700 1 --2- 172.21.15.3:0/3209476899 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f180c064f60 0x7f180c067410 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f18256a4700 1 --2- 172.21.15.3:0/3209476899 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1820160d20 0x7f1820152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f18256a4700 1 --2- 172.21.15.3:0/3209476899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18201608d0 0x7f182014e480 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f18256a4700 1 --2- 172.21.15.3:0/3209476899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1820153670 0x7f18200c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f18256a4700 1 -- 172.21.15.3:0/3209476899 >> 172.21.15.3:0/3209476899 conn(0x7f182006e5c0 msgr2=0x7f18200b7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:48.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f5fe1ffb700 1 -- 172.21.15.3:0/3925755914 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5fe4019090 con 0x7f5fec0c8460 2023-03-03T06:25:48.170 INFO:teuthology.orchestra.run.smithi003.stdout:55834574859 2023-03-03T06:25:48.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f18256a4700 1 -- 172.21.15.3:0/3209476899 shutdown_connections 2023-03-03T06:25:48.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.163+0000 7f18256a4700 1 -- 172.21.15.3:0/3209476899 wait complete. 2023-03-03T06:25:48.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5d7b29f700 1 -- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5d58064f30 msgr2=0x7f5d580673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5d7b29f700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5d58064f30 0x7f5d580673e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d64000c00 tx=0x7f5d64008030 comp rx=0 tx=0).stop 2023-03-03T06:25:48.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5d7b29f700 1 -- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d740cfb60 msgr2=0x7f5d7414b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5d7b29f700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d740cfb60 0x7f5d7414b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00cc20 tx=0x7f5d6c00cf30 comp rx=0 tx=0).stop 2023-03-03T06:25:48.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5d7b29f700 1 -- 172.21.15.3:0/2135546716 shutdown_connections 2023-03-03T06:25:48.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5d7b29f700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5d58064f30 0x7f5d580673e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5d7b29f700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d741608d0 0x7f5d740b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5d7b29f700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d74153670 0x7f5d740b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5d7b29f700 1 --2- 172.21.15.3:0/2135546716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d740cfb60 0x7f5d7414b0a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5d7b29f700 1 -- 172.21.15.3:0/2135546716 >> 172.21.15.3:0/2135546716 conn(0x7f5d7406e5c0 msgr2=0x7f5d740d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:48.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5d7b29f700 1 -- 172.21.15.3:0/2135546716 shutdown_connections 2023-03-03T06:25:48.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5d7b29f700 1 -- 172.21.15.3:0/2135546716 wait complete. 2023-03-03T06:25:48.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5fd0064f60 msgr2=0x7f5fd0067410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5ff38e6700 1 --2- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5fd0064f60 0x7f5fd0067410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009c60 tx=0x7f5fe8008030 comp rx=0 tx=0).stop 2023-03-03T06:25:48.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fec0c8460 msgr2=0x7f5fec0c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.167+0000 7f5ff38e6700 1 --2- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fec0c8460 0x7f5fec0c8e60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400be20 tx=0x7f5fe4009ed0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.171+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3925755914 shutdown_connections 2023-03-03T06:25:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.171+0000 7f5ff38e6700 1 --2- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5fd0064f60 0x7f5fd0067410 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.171+0000 7f5ff38e6700 1 --2- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fec0c8460 0x7f5fec0c8e60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.171+0000 7f5ff38e6700 1 --2- 172.21.15.3:0/3925755914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5fec160d70 0x7f5fec0c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.171+0000 7f5ff38e6700 1 --2- 172.21.15.3:0/3925755914 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5fec0b8380 0x7f5fec15b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.171+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3925755914 >> 172.21.15.3:0/3925755914 conn(0x7f5fec06e5c0 msgr2=0x7f5fec0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.171+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3925755914 shutdown_connections 2023-03-03T06:25:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.171+0000 7f5ff38e6700 1 -- 172.21.15.3:0/3925755914 wait complete. 2023-03-03T06:25:48.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.179+0000 7fa3b09f2700 1 -- 172.21.15.3:0/1182566552 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa3a4005770 con 0x7fa3a40e0560 2023-03-03T06:25:48.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.179+0000 7fa39affd700 1 -- 172.21.15.3:0/1182566552 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa3a0054f40 con 0x7fa3a40e0560 2023-03-03T06:25:48.184 INFO:teuthology.orchestra.run.smithi003.stdout:55834574860 2023-03-03T06:25:48.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7f609b484700 1 -- 172.21.15.3:0/161358904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f607c005890 con 0x7f6080005020 2023-03-03T06:25:48.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7f608affd700 1 -- 172.21.15.3:0/161358904 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6084053bf0 con 0x7f6080005020 2023-03-03T06:25:48.186 INFO:teuthology.orchestra.run.smithi003.stdout:55834574860 2023-03-03T06:25:48.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7fa3b09f2700 1 -- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa38c064fb0 msgr2=0x7fa38c067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7fa3b09f2700 1 --2- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa38c064fb0 0x7fa38c067460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009730 tx=0x7fa39c006ca0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7fa3b09f2700 1 -- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3a40e0560 msgr2=0x7fa3a40d25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7fa3b09f2700 1 --2- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3a40e0560 0x7fa3a40d25c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009fd0 tx=0x7fa3a0004770 comp rx=0 tx=0).stop 2023-03-03T06:25:48.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7fa3b09f2700 1 -- 172.21.15.3:0/1182566552 shutdown_connections 2023-03-03T06:25:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7fa3b09f2700 1 --2- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa38c064fb0 0x7fa38c067460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7fa3b09f2700 1 --2- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3a40e1610 0x7fa3a40ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7fa3b09f2700 1 --2- 172.21.15.3:0/1182566552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3a40e0560 0x7fa3a40d25c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7fa3b09f2700 1 --2- 172.21.15.3:0/1182566552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3a404faa0 0x7fa3a404a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7fa3b09f2700 1 -- 172.21.15.3:0/1182566552 >> 172.21.15.3:0/1182566552 conn(0x7fa3a4051a20 msgr2=0x7fa3a4006350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7fa3b09f2700 1 -- 172.21.15.3:0/1182566552 shutdown_connections 2023-03-03T06:25:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7fa3b09f2700 1 -- 172.21.15.3:0/1182566552 wait complete. 2023-03-03T06:25:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7f609b484700 1 -- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6078064fb0 msgr2=0x7f6078067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7f609b484700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6078064fb0 0x7f6078067460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6094068530 tx=0x7f6094072040 comp rx=0 tx=0).stop 2023-03-03T06:25:48.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7f609b484700 1 -- 172.21.15.3:0/161358904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6080005020 msgr2=0x7f608c04eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7f609b484700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6080005020 0x7f608c04eef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f60840109f0 tx=0x7f6084010d00 comp rx=0 tx=0).stop 2023-03-03T06:25:48.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7f609b484700 1 -- 172.21.15.3:0/161358904 shutdown_connections 2023-03-03T06:25:48.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7f609b484700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6078064fb0 0x7f6078067460 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7f609b484700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f608c17c5a0 0x7f608c0d7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7f609b484700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f608c04b4a0 0x7f608c03c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7f609b484700 1 --2- 172.21.15.3:0/161358904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6080005020 0x7f608c04eef0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7f609b484700 1 -- 172.21.15.3:0/161358904 >> 172.21.15.3:0/161358904 conn(0x7f608c00c250 msgr2=0x7f608c0ea720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:48.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.183+0000 7f47690d8700 1 -- 172.21.15.3:0/2112932403 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f47440055c0 con 0x7f475c0365d0 2023-03-03T06:25:48.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f609b484700 1 -- 172.21.15.3:0/161358904 shutdown_connections 2023-03-03T06:25:48.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f609b484700 1 -- 172.21.15.3:0/161358904 wait complete. 2023-03-03T06:25:48.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f474b7fe700 1 -- 172.21.15.3:0/2112932403 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f476409e030 con 0x7f475c0365d0 2023-03-03T06:25:48.193 INFO:teuthology.orchestra.run.smithi003.stdout:55834574860 2023-03-03T06:25:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f47690d8700 1 -- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4750064fb0 msgr2=0x7f4750067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f47690d8700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4750064fb0 0x7f4750067460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f475c04ee50 tx=0x7f4758008040 comp rx=0 tx=0).stop 2023-03-03T06:25:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f47690d8700 1 -- 172.21.15.3:0/2112932403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f475c0365d0 msgr2=0x7f475c04e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f47690d8700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f475c0365d0 0x7f475c04e5b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4764063970 tx=0x7f476404e430 comp rx=0 tx=0).stop 2023-03-03T06:25:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f47690d8700 1 -- 172.21.15.3:0/2112932403 shutdown_connections 2023-03-03T06:25:48.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f47690d8700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4750064fb0 0x7f4750067460 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f47690d8700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f475c0520e0 0x7f475c184560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f47690d8700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f475c036a20 0x7f475c0dd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f47690d8700 1 --2- 172.21.15.3:0/2112932403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f475c0365d0 0x7f475c04e5b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f47690d8700 1 -- 172.21.15.3:0/2112932403 >> 172.21.15.3:0/2112932403 conn(0x7f475c0515d0 msgr2=0x7f475c054a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f47690d8700 1 -- 172.21.15.3:0/2112932403 shutdown_connections 2023-03-03T06:25:48.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.187+0000 7f47690d8700 1 -- 172.21.15.3:0/2112932403 wait complete. 2023-03-03T06:25:48.212 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2023-03-03T06:25:48.212 DEBUG:teuthology.parallel:result is None 2023-03-03T06:25:48.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.7 2023-03-03T06:25:48.216 DEBUG:teuthology.parallel:result is None 2023-03-03T06:25:48.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.0 2023-03-03T06:25:48.220 DEBUG:teuthology.parallel:result is None 2023-03-03T06:25:48.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2023-03-03T06:25:48.221 DEBUG:teuthology.parallel:result is None 2023-03-03T06:25:48.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.4 2023-03-03T06:25:48.223 DEBUG:teuthology.parallel:result is None 2023-03-03T06:25:48.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.5 2023-03-03T06:25:48.227 DEBUG:teuthology.parallel:result is None 2023-03-03T06:25:48.227 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-03-03T06:25:48.227 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-03T06:25:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.467+0000 7fd74f59e700 1 Processor -- start 2023-03-03T06:25:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.467+0000 7fd74f59e700 1 -- start start 2023-03-03T06:25:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd750158440 0x7fd7500c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74f59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7500cdd30 con 0x7fd75015c750 2023-03-03T06:25:48.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7500ca4d0 con 0x7fd7500aee70 2023-03-03T06:25:48.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74f59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7500cf160 con 0x7fd750158440 2023-03-03T06:25:48.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd75015c750 0x7fd7500ce1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33786/0 (socket says 172.21.15.3:33786) 2023-03-03T06:25:48.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74ed9d700 1 -- 172.21.15.3:0/1668082283 learned_addr learned my addr 172.21.15.3:0/1668082283 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:48.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74e59c700 1 --2- 172.21.15.3:0/1668082283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd750158440 0x7fd7500c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:48.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1170405485 0 0) 0x7fd7500cdd30 con 0x7fd75015c750 2023-03-03T06:25:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd73c0035d0 con 0x7fd75015c750 2023-03-03T06:25:48.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1045991443 0 0) 0x7fd73c0035d0 con 0x7fd75015c750 2023-03-03T06:25:48.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd750158440 msgr2=0x7fd7500c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:48.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 --2- 172.21.15.3:0/1668082283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd750158440 0x7fd7500c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 >> v1:172.21.15.32:6789/0 conn(0x7fd7500aee70 legacy=0x7fd75015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7500cd6e0 con 0x7fd75015c750 2023-03-03T06:25:48.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7fd7480019f0 con 0x7fd75015c750 2023-03-03T06:25:48.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 >> v1:172.21.15.3:6789/0 conn(0x7fd75015c750 legacy=0x7fd7500ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 --2- 172.21.15.3:0/1668082283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd750158440 0x7fd73c003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 --2- 172.21.15.3:0/1668082283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c003fd0 0x7fd73c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 --2- 172.21.15.3:0/1668082283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c004db0 0x7fd73c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7500cf160 con 0x7fd73c003fd0 2023-03-03T06:25:48.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c00bb10 con 0x7fd73c004db0 2023-03-03T06:25:48.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c00dee0 con 0x7fd750158440 2023-03-03T06:25:48.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd745d9b700 1 --2- 172.21.15.3:0/1668082283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c003fd0 0x7fd73c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd74e59c700 1 --2- 172.21.15.3:0/1668082283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd750158440 0x7fd73c003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd745d9b700 1 -- 172.21.15.3:0/1668082283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd750158440 msgr2=0x7fd73c003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:48.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd745d9b700 1 --2- 172.21.15.3:0/1668082283 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd750158440 0x7fd73c003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.471+0000 7fd745d9b700 1 -- 172.21.15.3:0/1668082283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c004db0 msgr2=0x7fd73c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:48.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.475+0000 7fd745d9b700 1 --2- 172.21.15.3:0/1668082283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c004db0 0x7fd73c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.475+0000 7fd745d9b700 1 -- 172.21.15.3:0/1668082283 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c0035d0 con 0x7fd73c003fd0 2023-03-03T06:25:48.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.475+0000 7fd745d9b700 1 --2- 172.21.15.3:0/1668082283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c003fd0 0x7fd73c004870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd738007fd0 tx=0x7fd73c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37de260cd3edef0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:48.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.475+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7380052d0 con 0x7fd73c003fd0 2023-03-03T06:25:48.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.475+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd738005430 con 0x7fd73c003fd0 2023-03-03T06:25:48.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.475+0000 7fd74dd9b700 1 -- 172.21.15.3:0/1668082283 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7380055a0 con 0x7fd73c003fd0 2023-03-03T06:25:48.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.475+0000 7fd74f59e700 1 -- 172.21.15.3:0/1668082283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c003fd0 msgr2=0x7fd73c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.475+0000 7fd74f59e700 1 --2- 172.21.15.3:0/1668082283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c003fd0 0x7fd73c004870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd738007fd0 tx=0x7fd73c0058a0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.475+0000 7fd74ed9d700 1 -- 172.21.15.3:0/1668082283 reap_dead start 2023-03-03T06:25:48.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.475+0000 7fd74f59e700 1 -- 172.21.15.3:0/1668082283 shutdown_connections 2023-03-03T06:25:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.475+0000 7fd74f59e700 1 -- 172.21.15.3:0/1668082283 >> 172.21.15.3:0/1668082283 conn(0x7fd750070a30 msgr2=0x7fd750070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.475+0000 7fd74f59e700 1 -- 172.21.15.3:0/1668082283 shutdown_connections 2023-03-03T06:25:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.475+0000 7fd74f59e700 1 -- 172.21.15.3:0/1668082283 wait complete. 2023-03-03T06:25:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.479+0000 7fd74f59e700 1 Processor -- start 2023-03-03T06:25:48.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.479+0000 7fd74f59e700 1 -- start start 2023-03-03T06:25:48.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.479+0000 7fd74f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c003fd0 0x7fd750152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.479+0000 7fd74f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7501f4d20 0x7fd7501f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.479+0000 7fd74f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7501f1350 0x7fd75006e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.479+0000 7fd74f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7501f5190 con 0x7fd7501f1350 2023-03-03T06:25:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.479+0000 7fd74f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7501f5470 con 0x7fd73c003fd0 2023-03-03T06:25:48.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.479+0000 7fd74f59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7501f5750 con 0x7fd7501f4d20 2023-03-03T06:25:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.479+0000 7fd74ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7501f1350 0x7fd75006e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.479+0000 7fd74ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7501f1350 0x7fd75006e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43740/0 (socket says 172.21.15.3:43740) 2023-03-03T06:25:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.479+0000 7fd74ed9d700 1 -- 172.21.15.3:0/3398141733 learned_addr learned my addr 172.21.15.3:0/3398141733 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:48.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.479+0000 7fd745d9b700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7501f4d20 0x7fd7501f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:48.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.479+0000 7fd74e59c700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c003fd0 0x7fd750152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:48.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.483+0000 7fd74ed9d700 1 -- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7501f4d20 msgr2=0x7fd7501f0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.483+0000 7fd74ed9d700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7501f4d20 0x7fd7501f0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.483+0000 7fd74ed9d700 1 -- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c003fd0 msgr2=0x7fd750152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.483+0000 7fd74ed9d700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c003fd0 0x7fd750152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.483+0000 7fd74ed9d700 1 -- 172.21.15.3:0/3398141733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7500ca7b0 con 0x7fd7501f1350 2023-03-03T06:25:48.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.483+0000 7fd74e59c700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c003fd0 0x7fd750152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:48.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.483+0000 7fd74ed9d700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7501f1350 0x7fd75006e5c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd748001560 tx=0x7fd74800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:48.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.483+0000 7fd7477fe700 1 -- 172.21.15.3:0/3398141733 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74800ad30 con 0x7fd7501f1350 2023-03-03T06:25:48.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.483+0000 7fd7477fe700 1 -- 172.21.15.3:0/3398141733 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74800ae90 con 0x7fd7501f1350 2023-03-03T06:25:48.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.483+0000 7fd7477fe700 1 -- 172.21.15.3:0/3398141733 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748007310 con 0x7fd7501f1350 2023-03-03T06:25:48.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.483+0000 7fd74f59e700 1 -- 172.21.15.3:0/3398141733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7500cd9c0 con 0x7fd7501f1350 2023-03-03T06:25:48.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.483+0000 7fd74f59e700 1 -- 172.21.15.3:0/3398141733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75015cd90 con 0x7fd7501f1350 2023-03-03T06:25:48.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.483+0000 7fd74f59e700 1 -- 172.21.15.3:0/3398141733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd750073e90 con 0x7fd7501f1350 2023-03-03T06:25:48.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.487+0000 7fd7477fe700 1 -- 172.21.15.3:0/3398141733 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd748010070 con 0x7fd7501f1350 2023-03-03T06:25:48.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.487+0000 7fd7477fe700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd730064f10 0x7fd7300673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.487+0000 7fd7477fe700 1 -- 172.21.15.3:0/3398141733 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd748081a00 con 0x7fd7501f1350 2023-03-03T06:25:48.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.491+0000 7fd74e59c700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd730064f10 0x7fd7300673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:48.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.491+0000 7fd74e59c700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd730064f10 0x7fd7300673c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd740009b80 tx=0x7fd740008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:48.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.495+0000 7fd7477fe700 1 -- 172.21.15.3:0/3398141733 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd74804f760 con 0x7fd7501f1350 2023-03-03T06:25:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.659+0000 7fd74f59e700 1 -- 172.21.15.3:0/3398141733 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7500ccf10 con 0x7fd730064f10 2023-03-03T06:25:48.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.679+0000 7fd7477fe700 1 -- 172.21.15.3:0/3398141733 <== mgr.4105 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367170 (secure 0 0 0) 0x7fd7500ccf10 con 0x7fd730064f10 2023-03-03T06:25:48.689 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:48.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.687+0000 7fd74f59e700 1 -- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd730064f10 msgr2=0x7fd7300673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.687+0000 7fd74f59e700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd730064f10 0x7fd7300673c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd740009b80 tx=0x7fd740008040 comp rx=0 tx=0).stop 2023-03-03T06:25:48.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.687+0000 7fd74f59e700 1 -- 172.21.15.3:0/3398141733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7501f1350 msgr2=0x7fd75006e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.687+0000 7fd74f59e700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7501f1350 0x7fd75006e5c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd748001560 tx=0x7fd74800a340 comp rx=0 tx=0).stop 2023-03-03T06:25:48.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.687+0000 7fd74f59e700 1 -- 172.21.15.3:0/3398141733 shutdown_connections 2023-03-03T06:25:48.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.687+0000 7fd74f59e700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd730064f10 0x7fd7300673c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.687+0000 7fd74f59e700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd7501f1350 0x7fd75006e5c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.687+0000 7fd74f59e700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd7501f4d20 0x7fd7501f0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.691+0000 7fd74f59e700 1 --2- 172.21.15.3:0/3398141733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd73c003fd0 0x7fd750152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.691+0000 7fd74f59e700 1 -- 172.21.15.3:0/3398141733 >> 172.21.15.3:0/3398141733 conn(0x7fd750070a30 msgr2=0x7fd75006cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:48.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.691+0000 7fd74f59e700 1 -- 172.21.15.3:0/3398141733 shutdown_connections 2023-03-03T06:25:48.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.691+0000 7fd74f59e700 1 -- 172.21.15.3:0/3398141733 wait complete. 2023-03-03T06:25:48.697 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-03-03T06:25:48.725 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2023-03-03T06:25:48.410717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594895,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":84232,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":80576,"kb_avail":754890488,"statfs":{"total":773094113280,"available":773007859712,"internally_reserved":0,"allocated":3678208,"data_stored":2186366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":82509824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005690"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461806+0000","last_change":"2023-03-03T06:25:18.375133+0000","last_active":"2023-03-03T06:25:19.461806+0000","last_peered":"2023-03-03T06:25:19.461806+0000","last_clean":"2023-03-03T06:25:19.461806+0000","last_became_active":"2023-03-03T06:25:18.374500+0000","last_became_peered":"2023-03-03T06:25:18.374500+0000","last_unstale":"2023-03-03T06:25:19.461806+0000","last_undegraded":"2023-03-03T06:25:19.461806+0000","last_fullsized":"2023-03-03T06:25:19.461806+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:30.336080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343444+0000","last_change":"2023-03-03T06:25:13.301166+0000","last_active":"2023-03-03T06:25:18.343444+0000","last_peered":"2023-03-03T06:25:18.343444+0000","last_clean":"2023-03-03T06:25:18.343444+0000","last_became_active":"2023-03-03T06:25:13.300749+0000","last_became_peered":"2023-03-03T06:25:13.300749+0000","last_unstale":"2023-03-03T06:25:18.343444+0000","last_undegraded":"2023-03-03T06:25:18.343444+0000","last_fullsized":"2023-03-03T06:25:18.343444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335467+0000","last_change":"2023-03-03T06:25:13.839287+0000","last_active":"2023-03-03T06:25:18.335467+0000","last_peered":"2023-03-03T06:25:18.335467+0000","last_clean":"2023-03-03T06:25:18.335467+0000","last_became_active":"2023-03-03T06:25:13.838935+0000","last_became_peered":"2023-03-03T06:25:13.838935+0000","last_unstale":"2023-03-03T06:25:18.335467+0000","last_undegraded":"2023-03-03T06:25:18.335467+0000","last_fullsized":"2023-03-03T06:25:18.335467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:48:01.152785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.876352+0000","last_change":"2023-03-03T06:25:18.362828+0000","last_active":"2023-03-03T06:25:18.876352+0000","last_peered":"2023-03-03T06:25:18.876352+0000","last_clean":"2023-03-03T06:25:18.876352+0000","last_became_active":"2023-03-03T06:25:18.362423+0000","last_became_peered":"2023-03-03T06:25:18.362423+0000","last_unstale":"2023-03-03T06:25:18.876352+0000","last_undegraded":"2023-03-03T06:25:18.876352+0000","last_fullsized":"2023-03-03T06:25:18.876352+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:04:44.947509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.339522+0000","last_change":"2023-03-03T06:25:13.307540+0000","last_active":"2023-03-03T06:25:18.339522+0000","last_peered":"2023-03-03T06:25:18.339522+0000","last_clean":"2023-03-03T06:25:18.339522+0000","last_became_active":"2023-03-03T06:25:13.307229+0000","last_became_peered":"2023-03-03T06:25:13.307229+0000","last_unstale":"2023-03-03T06:25:18.339522+0000","last_undegraded":"2023-03-03T06:25:18.339522+0000","last_fullsized":"2023-03-03T06:25:18.339522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:09:34.703888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337997+0000","last_change":"2023-03-03T06:25:13.832118+0000","last_active":"2023-03-03T06:25:18.337997+0000","last_peered":"2023-03-03T06:25:18.337997+0000","last_clean":"2023-03-03T06:25:18.337997+0000","last_became_active":"2023-03-03T06:25:13.831744+0000","last_became_peered":"2023-03-03T06:25:13.831744+0000","last_unstale":"2023-03-03T06:25:18.337997+0000","last_undegraded":"2023-03-03T06:25:18.337997+0000","last_fullsized":"2023-03-03T06:25:18.337997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.878930+0000","last_change":"2023-03-03T06:25:18.367879+0000","last_active":"2023-03-03T06:25:18.878930+0000","last_peered":"2023-03-03T06:25:18.878930+0000","last_clean":"2023-03-03T06:25:18.878930+0000","last_became_active":"2023-03-03T06:25:18.367232+0000","last_became_peered":"2023-03-03T06:25:18.367232+0000","last_unstale":"2023-03-03T06:25:18.878930+0000","last_undegraded":"2023-03-03T06:25:18.878930+0000","last_fullsized":"2023-03-03T06:25:18.878930+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:43:20.174095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352877+0000","last_change":"2023-03-03T06:25:13.301087+0000","last_active":"2023-03-03T06:25:18.352877+0000","last_peered":"2023-03-03T06:25:18.352877+0000","last_clean":"2023-03-03T06:25:18.352877+0000","last_became_active":"2023-03-03T06:25:13.300837+0000","last_became_peered":"2023-03-03T06:25:13.300837+0000","last_unstale":"2023-03-03T06:25:18.352877+0000","last_undegraded":"2023-03-03T06:25:18.352877+0000","last_fullsized":"2023-03-03T06:25:18.352877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334901+0000","last_change":"2023-03-03T06:25:13.830954+0000","last_active":"2023-03-03T06:25:18.334901+0000","last_peered":"2023-03-03T06:25:18.334901+0000","last_clean":"2023-03-03T06:25:18.334901+0000","last_became_active":"2023-03-03T06:25:13.830614+0000","last_became_peered":"2023-03-03T06:25:13.830614+0000","last_unstale":"2023-03-03T06:25:18.334901+0000","last_undegraded":"2023-03-03T06:25:18.334901+0000","last_fullsized":"2023-03-03T06:25:18.334901+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.463051+0000","last_change":"2023-03-03T06:25:18.366781+0000","last_active":"2023-03-03T06:25:19.463051+0000","last_peered":"2023-03-03T06:25:19.463051+0000","last_clean":"2023-03-03T06:25:19.463051+0000","last_became_active":"2023-03-03T06:25:18.366175+0000","last_became_peered":"2023-03-03T06:25:18.366175+0000","last_unstale":"2023-03-03T06:25:19.463051+0000","last_undegraded":"2023-03-03T06:25:19.463051+0000","last_fullsized":"2023-03-03T06:25:19.463051+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:30.336080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352547+0000","last_change":"2023-03-03T06:25:13.300495+0000","last_active":"2023-03-03T06:25:18.352547+0000","last_peered":"2023-03-03T06:25:18.352547+0000","last_clean":"2023-03-03T06:25:18.352547+0000","last_became_active":"2023-03-03T06:25:13.300180+0000","last_became_peered":"2023-03-03T06:25:13.300180+0000","last_unstale":"2023-03-03T06:25:18.352547+0000","last_undegraded":"2023-03-03T06:25:18.352547+0000","last_fullsized":"2023-03-03T06:25:18.352547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335053+0000","last_change":"2023-03-03T06:25:13.830988+0000","last_active":"2023-03-03T06:25:18.335053+0000","last_peered":"2023-03-03T06:25:18.335053+0000","last_clean":"2023-03-03T06:25:18.335053+0000","last_became_active":"2023-03-03T06:25:13.826124+0000","last_became_peered":"2023-03-03T06:25:13.826124+0000","last_unstale":"2023-03-03T06:25:18.335053+0000","last_undegraded":"2023-03-03T06:25:18.335053+0000","last_fullsized":"2023-03-03T06:25:18.335053+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461936+0000","last_change":"2023-03-03T06:25:18.372519+0000","last_active":"2023-03-03T06:25:19.461936+0000","last_peered":"2023-03-03T06:25:19.461936+0000","last_clean":"2023-03-03T06:25:19.461936+0000","last_became_active":"2023-03-03T06:25:18.372087+0000","last_became_peered":"2023-03-03T06:25:18.372087+0000","last_unstale":"2023-03-03T06:25:19.461936+0000","last_undegraded":"2023-03-03T06:25:19.461936+0000","last_fullsized":"2023-03-03T06:25:19.461936+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:04.336725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344076+0000","last_change":"2023-03-03T06:25:13.308494+0000","last_active":"2023-03-03T06:25:18.344076+0000","last_peered":"2023-03-03T06:25:18.344076+0000","last_clean":"2023-03-03T06:25:18.344076+0000","last_became_active":"2023-03-03T06:25:13.308110+0000","last_became_peered":"2023-03-03T06:25:13.308110+0000","last_unstale":"2023-03-03T06:25:18.344076+0000","last_undegraded":"2023-03-03T06:25:18.344076+0000","last_fullsized":"2023-03-03T06:25:18.344076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.336445+0000","last_change":"2023-03-03T06:25:13.833412+0000","last_active":"2023-03-03T06:25:18.336445+0000","last_peered":"2023-03-03T06:25:18.336445+0000","last_clean":"2023-03-03T06:25:18.336445+0000","last_became_active":"2023-03-03T06:25:13.833138+0000","last_became_peered":"2023-03-03T06:25:13.833138+0000","last_unstale":"2023-03-03T06:25:18.336445+0000","last_undegraded":"2023-03-03T06:25:18.336445+0000","last_fullsized":"2023-03-03T06:25:18.336445+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:16:17.602774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.876849+0000","last_change":"2023-03-03T06:25:18.354570+0000","last_active":"2023-03-03T06:25:18.876849+0000","last_peered":"2023-03-03T06:25:18.876849+0000","last_clean":"2023-03-03T06:25:18.876849+0000","last_became_active":"2023-03-03T06:25:18.354128+0000","last_became_peered":"2023-03-03T06:25:18.354128+0000","last_unstale":"2023-03-03T06:25:18.876849+0000","last_undegraded":"2023-03-03T06:25:18.876849+0000","last_fullsized":"2023-03-03T06:25:18.876849+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:43:00.515957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335127+0000","last_change":"2023-03-03T06:25:13.307789+0000","last_active":"2023-03-03T06:25:18.335127+0000","last_peered":"2023-03-03T06:25:18.335127+0000","last_clean":"2023-03-03T06:25:18.335127+0000","last_became_active":"2023-03-03T06:25:13.307478+0000","last_became_peered":"2023-03-03T06:25:13.307478+0000","last_unstale":"2023-03-03T06:25:18.335127+0000","last_undegraded":"2023-03-03T06:25:18.335127+0000","last_fullsized":"2023-03-03T06:25:18.335127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:11:32.033289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.339285+0000","last_change":"2023-03-03T06:25:13.830797+0000","last_active":"2023-03-03T06:25:18.339285+0000","last_peered":"2023-03-03T06:25:18.339285+0000","last_clean":"2023-03-03T06:25:18.339285+0000","last_became_active":"2023-03-03T06:25:13.829482+0000","last_became_peered":"2023-03-03T06:25:13.829482+0000","last_unstale":"2023-03-03T06:25:18.339285+0000","last_undegraded":"2023-03-03T06:25:18.339285+0000","last_fullsized":"2023-03-03T06:25:18.339285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:48:01.152785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.459250+0000","last_change":"2023-03-03T06:25:18.358329+0000","last_active":"2023-03-03T06:25:19.459250+0000","last_peered":"2023-03-03T06:25:19.459250+0000","last_clean":"2023-03-03T06:25:19.459250+0000","last_became_active":"2023-03-03T06:25:18.357641+0000","last_became_peered":"2023-03-03T06:25:18.357641+0000","last_unstale":"2023-03-03T06:25:19.459250+0000","last_undegraded":"2023-03-03T06:25:19.459250+0000","last_fullsized":"2023-03-03T06:25:19.459250+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:59.071940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.359353+0000","last_change":"2023-03-03T06:25:13.312665+0000","last_active":"2023-03-03T06:25:18.359353+0000","last_peered":"2023-03-03T06:25:18.359353+0000","last_clean":"2023-03-03T06:25:18.359353+0000","last_became_active":"2023-03-03T06:25:13.312255+0000","last_became_peered":"2023-03-03T06:25:13.312255+0000","last_unstale":"2023-03-03T06:25:18.359353+0000","last_undegraded":"2023-03-03T06:25:18.359353+0000","last_fullsized":"2023-03-03T06:25:18.359353+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344419+0000","last_change":"2023-03-03T06:25:13.847886+0000","last_active":"2023-03-03T06:25:18.344419+0000","last_peered":"2023-03-03T06:25:18.344419+0000","last_clean":"2023-03-03T06:25:18.344419+0000","last_became_active":"2023-03-03T06:25:13.847620+0000","last_became_peered":"2023-03-03T06:25:13.847620+0000","last_unstale":"2023-03-03T06:25:18.344419+0000","last_undegraded":"2023-03-03T06:25:18.344419+0000","last_fullsized":"2023-03-03T06:25:18.344419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:48:01.152785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.875799+0000","last_change":"2023-03-03T06:25:18.374567+0000","last_active":"2023-03-03T06:25:18.875799+0000","last_peered":"2023-03-03T06:25:18.875799+0000","last_clean":"2023-03-03T06:25:18.875799+0000","last_became_active":"2023-03-03T06:25:18.374172+0000","last_became_peered":"2023-03-03T06:25:18.374172+0000","last_unstale":"2023-03-03T06:25:18.875799+0000","last_undegraded":"2023-03-03T06:25:18.875799+0000","last_fullsized":"2023-03-03T06:25:18.875799+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:11:32.109566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341581+0000","last_change":"2023-03-03T06:25:13.310405+0000","last_active":"2023-03-03T06:25:18.341581+0000","last_peered":"2023-03-03T06:25:18.341581+0000","last_clean":"2023-03-03T06:25:18.341581+0000","last_became_active":"2023-03-03T06:25:13.310029+0000","last_became_peered":"2023-03-03T06:25:13.310029+0000","last_unstale":"2023-03-03T06:25:18.341581+0000","last_undegraded":"2023-03-03T06:25:18.341581+0000","last_fullsized":"2023-03-03T06:25:18.341581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:16:16.586666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343606+0000","last_change":"2023-03-03T06:25:13.839392+0000","last_active":"2023-03-03T06:25:18.343606+0000","last_peered":"2023-03-03T06:25:18.343606+0000","last_clean":"2023-03-03T06:25:18.343606+0000","last_became_active":"2023-03-03T06:25:13.839078+0000","last_became_peered":"2023-03-03T06:25:13.839078+0000","last_unstale":"2023-03-03T06:25:18.343606+0000","last_undegraded":"2023-03-03T06:25:18.343606+0000","last_fullsized":"2023-03-03T06:25:18.343606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:42:30.992559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341473+0000","last_change":"2023-03-03T06:25:13.836064+0000","last_active":"2023-03-03T06:25:18.341473+0000","last_peered":"2023-03-03T06:25:18.341473+0000","last_clean":"2023-03-03T06:25:18.341473+0000","last_became_active":"2023-03-03T06:25:13.835743+0000","last_became_peered":"2023-03-03T06:25:13.835743+0000","last_unstale":"2023-03-03T06:25:18.341473+0000","last_undegraded":"2023-03-03T06:25:18.341473+0000","last_fullsized":"2023-03-03T06:25:18.341473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:52:13.551129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343523+0000","last_change":"2023-03-03T06:25:13.293588+0000","last_active":"2023-03-03T06:25:18.343523+0000","last_peered":"2023-03-03T06:25:18.343523+0000","last_clean":"2023-03-03T06:25:18.343523+0000","last_became_active":"2023-03-03T06:25:13.292998+0000","last_became_peered":"2023-03-03T06:25:13.292998+0000","last_unstale":"2023-03-03T06:25:18.343523+0000","last_undegraded":"2023-03-03T06:25:18.343523+0000","last_fullsized":"2023-03-03T06:25:18.343523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:17:34.388274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.459022+0000","last_change":"2023-03-03T06:25:18.365923+0000","last_active":"2023-03-03T06:25:19.459022+0000","last_peered":"2023-03-03T06:25:19.459022+0000","last_clean":"2023-03-03T06:25:19.459022+0000","last_became_active":"2023-03-03T06:25:18.365397+0000","last_became_peered":"2023-03-03T06:25:18.365397+0000","last_unstale":"2023-03-03T06:25:19.459022+0000","last_undegraded":"2023-03-03T06:25:19.459022+0000","last_fullsized":"2023-03-03T06:25:19.459022+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:45:30.096214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337824+0000","last_change":"2023-03-03T06:25:13.837989+0000","last_active":"2023-03-03T06:25:18.337824+0000","last_peered":"2023-03-03T06:25:18.337824+0000","last_clean":"2023-03-03T06:25:18.337824+0000","last_became_active":"2023-03-03T06:25:13.837724+0000","last_became_peered":"2023-03-03T06:25:13.837724+0000","last_unstale":"2023-03-03T06:25:18.337824+0000","last_undegraded":"2023-03-03T06:25:18.337824+0000","last_fullsized":"2023-03-03T06:25:18.337824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353186+0000","last_change":"2023-03-03T06:25:13.302153+0000","last_active":"2023-03-03T06:25:18.353186+0000","last_peered":"2023-03-03T06:25:18.353186+0000","last_clean":"2023-03-03T06:25:18.353186+0000","last_became_active":"2023-03-03T06:25:13.301751+0000","last_became_peered":"2023-03-03T06:25:13.301751+0000","last_unstale":"2023-03-03T06:25:18.353186+0000","last_undegraded":"2023-03-03T06:25:18.353186+0000","last_fullsized":"2023-03-03T06:25:18.353186+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.462897+0000","last_change":"2023-03-03T06:25:18.366010+0000","last_active":"2023-03-03T06:25:19.462897+0000","last_peered":"2023-03-03T06:25:19.462897+0000","last_clean":"2023-03-03T06:25:19.462897+0000","last_became_active":"2023-03-03T06:25:18.362457+0000","last_became_peered":"2023-03-03T06:25:18.362457+0000","last_unstale":"2023-03-03T06:25:19.462897+0000","last_undegraded":"2023-03-03T06:25:19.462897+0000","last_fullsized":"2023-03-03T06:25:19.462897+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:17:02.442524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350698+0000","last_change":"2023-03-03T06:25:13.312520+0000","last_active":"2023-03-03T06:25:18.350698+0000","last_peered":"2023-03-03T06:25:18.350698+0000","last_clean":"2023-03-03T06:25:18.350698+0000","last_became_active":"2023-03-03T06:25:13.312105+0000","last_became_peered":"2023-03-03T06:25:13.312105+0000","last_unstale":"2023-03-03T06:25:18.350698+0000","last_undegraded":"2023-03-03T06:25:18.350698+0000","last_fullsized":"2023-03-03T06:25:18.350698+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:51:29.967970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358702+0000","last_change":"2023-03-03T06:25:13.835492+0000","last_active":"2023-03-03T06:25:18.358702+0000","last_peered":"2023-03-03T06:25:18.358702+0000","last_clean":"2023-03-03T06:25:18.358702+0000","last_became_active":"2023-03-03T06:25:13.835173+0000","last_became_peered":"2023-03-03T06:25:13.835173+0000","last_unstale":"2023-03-03T06:25:18.358702+0000","last_undegraded":"2023-03-03T06:25:18.358702+0000","last_fullsized":"2023-03-03T06:25:18.358702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467726+0000","last_change":"2023-03-03T06:25:18.362177+0000","last_active":"2023-03-03T06:25:19.467726+0000","last_peered":"2023-03-03T06:25:19.467726+0000","last_clean":"2023-03-03T06:25:19.467726+0000","last_became_active":"2023-03-03T06:25:18.361833+0000","last_became_peered":"2023-03-03T06:25:18.361833+0000","last_unstale":"2023-03-03T06:25:19.467726+0000","last_undegraded":"2023-03-03T06:25:19.467726+0000","last_fullsized":"2023-03-03T06:25:19.467726+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:04:44.947509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:32.598145+0000","last_change":"2023-03-03T06:25:13.302558+0000","last_active":"2023-03-03T06:25:32.598145+0000","last_peered":"2023-03-03T06:25:32.598145+0000","last_clean":"2023-03-03T06:25:32.598145+0000","last_became_active":"2023-03-03T06:25:13.301983+0000","last_became_peered":"2023-03-03T06:25:13.301983+0000","last_unstale":"2023-03-03T06:25:32.598145+0000","last_undegraded":"2023-03-03T06:25:32.598145+0000","last_fullsized":"2023-03-03T06:25:32.598145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:34:41.158814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3745,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334685+0000","last_change":"2023-03-03T06:25:13.835359+0000","last_active":"2023-03-03T06:25:18.334685+0000","last_peered":"2023-03-03T06:25:18.334685+0000","last_clean":"2023-03-03T06:25:18.334685+0000","last_became_active":"2023-03-03T06:25:13.835103+0000","last_became_peered":"2023-03-03T06:25:13.835103+0000","last_unstale":"2023-03-03T06:25:18.334685+0000","last_undegraded":"2023-03-03T06:25:18.334685+0000","last_fullsized":"2023-03-03T06:25:18.334685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461833+0000","last_change":"2023-03-03T06:25:18.376174+0000","last_active":"2023-03-03T06:25:19.461833+0000","last_peered":"2023-03-03T06:25:19.461833+0000","last_clean":"2023-03-03T06:25:19.461833+0000","last_became_active":"2023-03-03T06:25:18.375779+0000","last_became_peered":"2023-03-03T06:25:18.375779+0000","last_unstale":"2023-03-03T06:25:19.461833+0000","last_undegraded":"2023-03-03T06:25:19.461833+0000","last_fullsized":"2023-03-03T06:25:19.461833+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:40:49.542836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335280+0000","last_change":"2023-03-03T06:25:13.837277+0000","last_active":"2023-03-03T06:25:18.335280+0000","last_peered":"2023-03-03T06:25:18.335280+0000","last_clean":"2023-03-03T06:25:18.335280+0000","last_became_active":"2023-03-03T06:25:13.836510+0000","last_became_peered":"2023-03-03T06:25:13.836510+0000","last_unstale":"2023-03-03T06:25:18.335280+0000","last_undegraded":"2023-03-03T06:25:18.335280+0000","last_fullsized":"2023-03-03T06:25:18.335280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343682+0000","last_change":"2023-03-03T06:25:07.205700+0000","last_active":"2023-03-03T06:25:18.343682+0000","last_peered":"2023-03-03T06:25:18.343682+0000","last_clean":"2023-03-03T06:25:18.343682+0000","last_became_active":"2023-03-03T06:25:07.205269+0000","last_became_peered":"2023-03-03T06:25:07.205269+0000","last_unstale":"2023-03-03T06:25:18.343682+0000","last_undegraded":"2023-03-03T06:25:18.343682+0000","last_fullsized":"2023-03-03T06:25:18.343682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:17:53.442570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344712+0000","last_change":"2023-03-03T06:25:13.309537+0000","last_active":"2023-03-03T06:25:18.344712+0000","last_peered":"2023-03-03T06:25:18.344712+0000","last_clean":"2023-03-03T06:25:18.344712+0000","last_became_active":"2023-03-03T06:25:13.309208+0000","last_became_peered":"2023-03-03T06:25:13.309208+0000","last_unstale":"2023-03-03T06:25:18.344712+0000","last_undegraded":"2023-03-03T06:25:18.344712+0000","last_fullsized":"2023-03-03T06:25:18.344712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:11:45.788884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467622+0000","last_change":"2023-03-03T06:25:18.367070+0000","last_active":"2023-03-03T06:25:19.467622+0000","last_peered":"2023-03-03T06:25:19.467622+0000","last_clean":"2023-03-03T06:25:19.467622+0000","last_became_active":"2023-03-03T06:25:18.366674+0000","last_became_peered":"2023-03-03T06:25:18.366674+0000","last_unstale":"2023-03-03T06:25:19.467622+0000","last_undegraded":"2023-03-03T06:25:19.467622+0000","last_fullsized":"2023-03-03T06:25:19.467622+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:43:20.174095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341556+0000","last_change":"2023-03-03T06:25:13.837256+0000","last_active":"2023-03-03T06:25:18.341556+0000","last_peered":"2023-03-03T06:25:18.341556+0000","last_clean":"2023-03-03T06:25:18.341556+0000","last_became_active":"2023-03-03T06:25:13.836942+0000","last_became_peered":"2023-03-03T06:25:13.836942+0000","last_unstale":"2023-03-03T06:25:18.341556+0000","last_undegraded":"2023-03-03T06:25:18.341556+0000","last_fullsized":"2023-03-03T06:25:18.341556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:05:53.754239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357705+0000","last_change":"2023-03-03T06:25:07.204903+0000","last_active":"2023-03-03T06:25:18.357705+0000","last_peered":"2023-03-03T06:25:18.357705+0000","last_clean":"2023-03-03T06:25:18.357705+0000","last_became_active":"2023-03-03T06:25:07.204601+0000","last_became_peered":"2023-03-03T06:25:07.204601+0000","last_unstale":"2023-03-03T06:25:18.357705+0000","last_undegraded":"2023-03-03T06:25:18.357705+0000","last_fullsized":"2023-03-03T06:25:18.357705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:18:36.947912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343364+0000","last_change":"2023-03-03T06:25:13.304543+0000","last_active":"2023-03-03T06:25:18.343364+0000","last_peered":"2023-03-03T06:25:18.343364+0000","last_clean":"2023-03-03T06:25:18.343364+0000","last_became_active":"2023-03-03T06:25:13.304142+0000","last_became_peered":"2023-03-03T06:25:13.304142+0000","last_unstale":"2023-03-03T06:25:18.343364+0000","last_undegraded":"2023-03-03T06:25:18.343364+0000","last_fullsized":"2023-03-03T06:25:18.343364+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.876556+0000","last_change":"2023-03-03T06:25:18.354139+0000","last_active":"2023-03-03T06:25:18.876556+0000","last_peered":"2023-03-03T06:25:18.876556+0000","last_clean":"2023-03-03T06:25:18.876556+0000","last_became_active":"2023-03-03T06:25:18.353543+0000","last_became_peered":"2023-03-03T06:25:18.353543+0000","last_unstale":"2023-03-03T06:25:18.876556+0000","last_undegraded":"2023-03-03T06:25:18.876556+0000","last_fullsized":"2023-03-03T06:25:18.876556+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:11:23.940316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343906+0000","last_change":"2023-03-03T06:25:13.838819+0000","last_active":"2023-03-03T06:25:18.343906+0000","last_peered":"2023-03-03T06:25:18.343906+0000","last_clean":"2023-03-03T06:25:18.343906+0000","last_became_active":"2023-03-03T06:25:13.838558+0000","last_became_peered":"2023-03-03T06:25:13.838558+0000","last_unstale":"2023-03-03T06:25:18.343906+0000","last_undegraded":"2023-03-03T06:25:18.343906+0000","last_fullsized":"2023-03-03T06:25:18.343906+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350807+0000","last_change":"2023-03-03T06:25:07.208473+0000","last_active":"2023-03-03T06:25:18.350807+0000","last_peered":"2023-03-03T06:25:18.350807+0000","last_clean":"2023-03-03T06:25:18.350807+0000","last_became_active":"2023-03-03T06:25:07.208041+0000","last_became_peered":"2023-03-03T06:25:07.208041+0000","last_unstale":"2023-03-03T06:25:18.350807+0000","last_undegraded":"2023-03-03T06:25:18.350807+0000","last_fullsized":"2023-03-03T06:25:18.350807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:22:19.852493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357626+0000","last_change":"2023-03-03T06:25:13.308259+0000","last_active":"2023-03-03T06:25:18.357626+0000","last_peered":"2023-03-03T06:25:18.357626+0000","last_clean":"2023-03-03T06:25:18.357626+0000","last_became_active":"2023-03-03T06:25:13.307902+0000","last_became_peered":"2023-03-03T06:25:13.307902+0000","last_unstale":"2023-03-03T06:25:18.357626+0000","last_undegraded":"2023-03-03T06:25:18.357626+0000","last_fullsized":"2023-03-03T06:25:18.357626+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.459074+0000","last_change":"2023-03-03T06:25:18.355878+0000","last_active":"2023-03-03T06:25:19.459074+0000","last_peered":"2023-03-03T06:25:19.459074+0000","last_clean":"2023-03-03T06:25:19.459074+0000","last_became_active":"2023-03-03T06:25:18.355460+0000","last_became_peered":"2023-03-03T06:25:18.355460+0000","last_unstale":"2023-03-03T06:25:19.459074+0000","last_undegraded":"2023-03-03T06:25:19.459074+0000","last_fullsized":"2023-03-03T06:25:19.459074+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:38:51.150150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358771+0000","last_change":"2023-03-03T06:25:13.839947+0000","last_active":"2023-03-03T06:25:18.358771+0000","last_peered":"2023-03-03T06:25:18.358771+0000","last_clean":"2023-03-03T06:25:18.358771+0000","last_became_active":"2023-03-03T06:25:13.839632+0000","last_became_peered":"2023-03-03T06:25:13.839632+0000","last_unstale":"2023-03-03T06:25:18.358771+0000","last_undegraded":"2023-03-03T06:25:18.358771+0000","last_fullsized":"2023-03-03T06:25:18.358771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357780+0000","last_change":"2023-03-03T06:25:07.201865+0000","last_active":"2023-03-03T06:25:18.357780+0000","last_peered":"2023-03-03T06:25:18.357780+0000","last_clean":"2023-03-03T06:25:18.357780+0000","last_became_active":"2023-03-03T06:25:07.201411+0000","last_became_peered":"2023-03-03T06:25:07.201411+0000","last_unstale":"2023-03-03T06:25:18.357780+0000","last_undegraded":"2023-03-03T06:25:18.357780+0000","last_fullsized":"2023-03-03T06:25:18.357780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:17:53.442570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352970+0000","last_change":"2023-03-03T06:25:13.312426+0000","last_active":"2023-03-03T06:25:18.352970+0000","last_peered":"2023-03-03T06:25:18.352970+0000","last_clean":"2023-03-03T06:25:18.352970+0000","last_became_active":"2023-03-03T06:25:13.312129+0000","last_became_peered":"2023-03-03T06:25:13.312129+0000","last_unstale":"2023-03-03T06:25:18.352970+0000","last_undegraded":"2023-03-03T06:25:18.352970+0000","last_fullsized":"2023-03-03T06:25:18.352970+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.875532+0000","last_change":"2023-03-03T06:25:18.365085+0000","last_active":"2023-03-03T06:25:18.875532+0000","last_peered":"2023-03-03T06:25:18.875532+0000","last_clean":"2023-03-03T06:25:18.875532+0000","last_became_active":"2023-03-03T06:25:18.364690+0000","last_became_peered":"2023-03-03T06:25:18.364690+0000","last_unstale":"2023-03-03T06:25:18.875532+0000","last_undegraded":"2023-03-03T06:25:18.875532+0000","last_fullsized":"2023-03-03T06:25:18.875532+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:30.336080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357548+0000","last_change":"2023-03-03T06:25:13.828855+0000","last_active":"2023-03-03T06:25:18.357548+0000","last_peered":"2023-03-03T06:25:18.357548+0000","last_clean":"2023-03-03T06:25:18.357548+0000","last_became_active":"2023-03-03T06:25:13.828546+0000","last_became_peered":"2023-03-03T06:25:13.828546+0000","last_unstale":"2023-03-03T06:25:18.357548+0000","last_undegraded":"2023-03-03T06:25:18.357548+0000","last_fullsized":"2023-03-03T06:25:18.357548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334824+0000","last_change":"2023-03-03T06:25:13.298859+0000","last_active":"2023-03-03T06:25:18.334824+0000","last_peered":"2023-03-03T06:25:18.334824+0000","last_clean":"2023-03-03T06:25:18.334824+0000","last_became_active":"2023-03-03T06:25:13.298586+0000","last_became_peered":"2023-03-03T06:25:13.298586+0000","last_unstale":"2023-03-03T06:25:18.334824+0000","last_undegraded":"2023-03-03T06:25:18.334824+0000","last_fullsized":"2023-03-03T06:25:18.334824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:11:45.788884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353089+0000","last_change":"2023-03-03T06:25:07.201941+0000","last_active":"2023-03-03T06:25:18.353089+0000","last_peered":"2023-03-03T06:25:18.353089+0000","last_clean":"2023-03-03T06:25:18.353089+0000","last_became_active":"2023-03-03T06:25:07.201437+0000","last_became_peered":"2023-03-03T06:25:07.201437+0000","last_unstale":"2023-03-03T06:25:18.353089+0000","last_undegraded":"2023-03-03T06:25:18.353089+0000","last_fullsized":"2023-03-03T06:25:18.353089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:48:52.145810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.875719+0000","last_change":"2023-03-03T06:25:18.374636+0000","last_active":"2023-03-03T06:25:18.875719+0000","last_peered":"2023-03-03T06:25:18.875719+0000","last_clean":"2023-03-03T06:25:18.875719+0000","last_became_active":"2023-03-03T06:25:18.373925+0000","last_became_peered":"2023-03-03T06:25:18.373925+0000","last_unstale":"2023-03-03T06:25:18.875719+0000","last_undegraded":"2023-03-03T06:25:18.875719+0000","last_fullsized":"2023-03-03T06:25:18.875719+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:07:38.561763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.359409+0000","last_change":"2023-03-03T06:25:13.833104+0000","last_active":"2023-03-03T06:25:18.359409+0000","last_peered":"2023-03-03T06:25:18.359409+0000","last_clean":"2023-03-03T06:25:18.359409+0000","last_became_active":"2023-03-03T06:25:13.832815+0000","last_became_peered":"2023-03-03T06:25:13.832815+0000","last_unstale":"2023-03-03T06:25:18.359409+0000","last_undegraded":"2023-03-03T06:25:18.359409+0000","last_fullsized":"2023-03-03T06:25:18.359409+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.356972+0000","last_change":"2023-03-03T06:25:13.298339+0000","last_active":"2023-03-03T06:25:18.356972+0000","last_peered":"2023-03-03T06:25:18.356972+0000","last_clean":"2023-03-03T06:25:18.356972+0000","last_became_active":"2023-03-03T06:25:13.297116+0000","last_became_peered":"2023-03-03T06:25:13.297116+0000","last_unstale":"2023-03-03T06:25:18.356972+0000","last_undegraded":"2023-03-03T06:25:18.356972+0000","last_fullsized":"2023-03-03T06:25:18.356972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335947+0000","last_change":"2023-03-03T06:25:07.437275+0000","last_active":"2023-03-03T06:25:18.335947+0000","last_peered":"2023-03-03T06:25:18.335947+0000","last_clean":"2023-03-03T06:25:18.335947+0000","last_became_active":"2023-03-03T06:25:07.436823+0000","last_became_peered":"2023-03-03T06:25:07.436823+0000","last_unstale":"2023-03-03T06:25:18.335947+0000","last_undegraded":"2023-03-03T06:25:18.335947+0000","last_fullsized":"2023-03-03T06:25:18.335947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:24:15.529041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217928+0000","last_change":"2023-03-03T06:25:18.361949+0000","last_active":"2023-03-03T06:25:19.217928+0000","last_peered":"2023-03-03T06:25:19.217928+0000","last_clean":"2023-03-03T06:25:19.217928+0000","last_became_active":"2023-03-03T06:25:18.361407+0000","last_became_peered":"2023-03-03T06:25:18.361407+0000","last_unstale":"2023-03-03T06:25:19.217928+0000","last_undegraded":"2023-03-03T06:25:19.217928+0000","last_fullsized":"2023-03-03T06:25:19.217928+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:59.071940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344511+0000","last_change":"2023-03-03T06:25:13.833788+0000","last_active":"2023-03-03T06:25:18.344511+0000","last_peered":"2023-03-03T06:25:18.344511+0000","last_clean":"2023-03-03T06:25:18.344511+0000","last_became_active":"2023-03-03T06:25:13.833532+0000","last_became_peered":"2023-03-03T06:25:13.833532+0000","last_unstale":"2023-03-03T06:25:18.344511+0000","last_undegraded":"2023-03-03T06:25:18.344511+0000","last_fullsized":"2023-03-03T06:25:18.344511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:59:50.232210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342831+0000","last_change":"2023-03-03T06:25:13.308357+0000","last_active":"2023-03-03T06:25:18.342831+0000","last_peered":"2023-03-03T06:25:18.342831+0000","last_clean":"2023-03-03T06:25:18.342831+0000","last_became_active":"2023-03-03T06:25:13.307981+0000","last_became_peered":"2023-03-03T06:25:13.307981+0000","last_unstale":"2023-03-03T06:25:18.342831+0000","last_undegraded":"2023-03-03T06:25:18.342831+0000","last_fullsized":"2023-03-03T06:25:18.342831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346873+0000","last_change":"2023-03-03T06:25:07.208585+0000","last_active":"2023-03-03T06:25:18.346873+0000","last_peered":"2023-03-03T06:25:18.346873+0000","last_clean":"2023-03-03T06:25:18.346873+0000","last_became_active":"2023-03-03T06:25:07.208257+0000","last_became_peered":"2023-03-03T06:25:07.208257+0000","last_unstale":"2023-03-03T06:25:18.346873+0000","last_undegraded":"2023-03-03T06:25:18.346873+0000","last_fullsized":"2023-03-03T06:25:18.346873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:11:52.893733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879358+0000","last_change":"2023-03-03T06:25:18.369960+0000","last_active":"2023-03-03T06:25:18.879358+0000","last_peered":"2023-03-03T06:25:18.879358+0000","last_clean":"2023-03-03T06:25:18.879358+0000","last_became_active":"2023-03-03T06:25:18.367846+0000","last_became_peered":"2023-03-03T06:25:18.367846+0000","last_unstale":"2023-03-03T06:25:18.879358+0000","last_undegraded":"2023-03-03T06:25:18.879358+0000","last_fullsized":"2023-03-03T06:25:18.879358+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:12:09.571057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342233+0000","last_change":"2023-03-03T06:25:13.857729+0000","last_active":"2023-03-03T06:25:18.342233+0000","last_peered":"2023-03-03T06:25:18.342233+0000","last_clean":"2023-03-03T06:25:18.342233+0000","last_became_active":"2023-03-03T06:25:13.857453+0000","last_became_peered":"2023-03-03T06:25:13.857453+0000","last_unstale":"2023-03-03T06:25:18.342233+0000","last_undegraded":"2023-03-03T06:25:18.342233+0000","last_fullsized":"2023-03-03T06:25:18.342233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:46:08.313482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.339103+0000","last_change":"2023-03-03T06:25:13.303419+0000","last_active":"2023-03-03T06:25:18.339103+0000","last_peered":"2023-03-03T06:25:18.339103+0000","last_clean":"2023-03-03T06:25:18.339103+0000","last_became_active":"2023-03-03T06:25:13.303084+0000","last_became_peered":"2023-03-03T06:25:13.303084+0000","last_unstale":"2023-03-03T06:25:18.339103+0000","last_undegraded":"2023-03-03T06:25:18.339103+0000","last_fullsized":"2023-03-03T06:25:18.339103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352430+0000","last_change":"2023-03-03T06:25:07.203115+0000","last_active":"2023-03-03T06:25:18.352430+0000","last_peered":"2023-03-03T06:25:18.352430+0000","last_clean":"2023-03-03T06:25:18.352430+0000","last_became_active":"2023-03-03T06:25:07.202705+0000","last_became_peered":"2023-03-03T06:25:07.202705+0000","last_unstale":"2023-03-03T06:25:18.352430+0000","last_undegraded":"2023-03-03T06:25:18.352430+0000","last_fullsized":"2023-03-03T06:25:18.352430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:11:32.938698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346951+0000","last_change":"2023-03-03T06:25:02.138688+0000","last_active":"2023-03-03T06:25:18.346951+0000","last_peered":"2023-03-03T06:25:18.346951+0000","last_clean":"2023-03-03T06:25:18.346951+0000","last_became_active":"2023-03-03T06:25:02.137945+0000","last_became_peered":"2023-03-03T06:25:02.137945+0000","last_unstale":"2023-03-03T06:25:18.346951+0000","last_undegraded":"2023-03-03T06:25:18.346951+0000","last_fullsized":"2023-03-03T06:25:18.346951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:00.679233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:00.679233+0000","last_clean_scrub_stamp":"2023-03-03T06:25:00.679233+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:24:46.304379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.463093+0000","last_change":"2023-03-03T06:25:18.368371+0000","last_active":"2023-03-03T06:25:19.463093+0000","last_peered":"2023-03-03T06:25:19.463093+0000","last_clean":"2023-03-03T06:25:19.463093+0000","last_became_active":"2023-03-03T06:25:18.367896+0000","last_became_peered":"2023-03-03T06:25:18.367896+0000","last_unstale":"2023-03-03T06:25:19.463093+0000","last_undegraded":"2023-03-03T06:25:19.463093+0000","last_fullsized":"2023-03-03T06:25:19.463093+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:43:20.174095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358378+0000","last_change":"2023-03-03T06:25:13.842311+0000","last_active":"2023-03-03T06:25:18.358378+0000","last_peered":"2023-03-03T06:25:18.358378+0000","last_clean":"2023-03-03T06:25:18.358378+0000","last_became_active":"2023-03-03T06:25:13.841952+0000","last_became_peered":"2023-03-03T06:25:13.841952+0000","last_unstale":"2023-03-03T06:25:18.358378+0000","last_undegraded":"2023-03-03T06:25:18.358378+0000","last_fullsized":"2023-03-03T06:25:18.358378+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:46:08.313482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341679+0000","last_change":"2023-03-03T06:25:13.306584+0000","last_active":"2023-03-03T06:25:18.341679+0000","last_peered":"2023-03-03T06:25:18.341679+0000","last_clean":"2023-03-03T06:25:18.341679+0000","last_became_active":"2023-03-03T06:25:13.306295+0000","last_became_peered":"2023-03-03T06:25:13.306295+0000","last_unstale":"2023-03-03T06:25:18.341679+0000","last_undegraded":"2023-03-03T06:25:18.341679+0000","last_fullsized":"2023-03-03T06:25:18.341679+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.878155+0000","last_change":"2023-03-03T06:25:18.361381+0000","last_active":"2023-03-03T06:25:18.878155+0000","last_peered":"2023-03-03T06:25:18.878155+0000","last_clean":"2023-03-03T06:25:18.878155+0000","last_became_active":"2023-03-03T06:25:18.360819+0000","last_became_peered":"2023-03-03T06:25:18.360819+0000","last_unstale":"2023-03-03T06:25:18.878155+0000","last_undegraded":"2023-03-03T06:25:18.878155+0000","last_fullsized":"2023-03-03T06:25:18.878155+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:39:21.927192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335650+0000","last_change":"2023-03-03T06:25:13.848629+0000","last_active":"2023-03-03T06:25:18.335650+0000","last_peered":"2023-03-03T06:25:18.335650+0000","last_clean":"2023-03-03T06:25:18.335650+0000","last_became_active":"2023-03-03T06:25:13.848353+0000","last_became_peered":"2023-03-03T06:25:13.848353+0000","last_unstale":"2023-03-03T06:25:18.335650+0000","last_undegraded":"2023-03-03T06:25:18.335650+0000","last_fullsized":"2023-03-03T06:25:18.335650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:59:50.232210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335310+0000","last_change":"2023-03-03T06:25:13.297292+0000","last_active":"2023-03-03T06:25:18.335310+0000","last_peered":"2023-03-03T06:25:18.335310+0000","last_clean":"2023-03-03T06:25:18.335310+0000","last_became_active":"2023-03-03T06:25:13.296692+0000","last_became_peered":"2023-03-03T06:25:13.296692+0000","last_unstale":"2023-03-03T06:25:18.335310+0000","last_undegraded":"2023-03-03T06:25:18.335310+0000","last_fullsized":"2023-03-03T06:25:18.335310+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:09:43.297428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879361+0000","last_change":"2023-03-03T06:25:18.369017+0000","last_active":"2023-03-03T06:25:18.879361+0000","last_peered":"2023-03-03T06:25:18.879361+0000","last_clean":"2023-03-03T06:25:18.879361+0000","last_became_active":"2023-03-03T06:25:18.366873+0000","last_became_peered":"2023-03-03T06:25:18.366873+0000","last_unstale":"2023-03-03T06:25:18.879361+0000","last_undegraded":"2023-03-03T06:25:18.879361+0000","last_fullsized":"2023-03-03T06:25:18.879361+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:17:02.442524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334965+0000","last_change":"2023-03-03T06:25:13.855863+0000","last_active":"2023-03-03T06:25:18.334965+0000","last_peered":"2023-03-03T06:25:18.334965+0000","last_clean":"2023-03-03T06:25:18.334965+0000","last_became_active":"2023-03-03T06:25:13.855565+0000","last_became_peered":"2023-03-03T06:25:13.855565+0000","last_unstale":"2023-03-03T06:25:18.334965+0000","last_undegraded":"2023-03-03T06:25:18.334965+0000","last_fullsized":"2023-03-03T06:25:18.334965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:45:26.049658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341408+0000","last_change":"2023-03-03T06:25:13.306690+0000","last_active":"2023-03-03T06:25:18.341408+0000","last_peered":"2023-03-03T06:25:18.341408+0000","last_clean":"2023-03-03T06:25:18.341408+0000","last_became_active":"2023-03-03T06:25:13.306348+0000","last_became_peered":"2023-03-03T06:25:13.306348+0000","last_unstale":"2023-03-03T06:25:18.341408+0000","last_undegraded":"2023-03-03T06:25:18.341408+0000","last_fullsized":"2023-03-03T06:25:18.341408+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461865+0000","last_change":"2023-03-03T06:25:18.373926+0000","last_active":"2023-03-03T06:25:19.461865+0000","last_peered":"2023-03-03T06:25:19.461865+0000","last_clean":"2023-03-03T06:25:19.461865+0000","last_became_active":"2023-03-03T06:25:18.373371+0000","last_became_peered":"2023-03-03T06:25:18.373371+0000","last_unstale":"2023-03-03T06:25:19.461865+0000","last_undegraded":"2023-03-03T06:25:19.461865+0000","last_fullsized":"2023-03-03T06:25:19.461865+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:17:02.442524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335386+0000","last_change":"2023-03-03T06:25:13.846359+0000","last_active":"2023-03-03T06:25:18.335386+0000","last_peered":"2023-03-03T06:25:18.335386+0000","last_clean":"2023-03-03T06:25:18.335386+0000","last_became_active":"2023-03-03T06:25:13.845966+0000","last_became_peered":"2023-03-03T06:25:13.845966+0000","last_unstale":"2023-03-03T06:25:18.335386+0000","last_undegraded":"2023-03-03T06:25:18.335386+0000","last_fullsized":"2023-03-03T06:25:18.335386+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:38:43.138896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341996+0000","last_change":"2023-03-03T06:25:13.310274+0000","last_active":"2023-03-03T06:25:18.341996+0000","last_peered":"2023-03-03T06:25:18.341996+0000","last_clean":"2023-03-03T06:25:18.341996+0000","last_became_active":"2023-03-03T06:25:13.310005+0000","last_became_peered":"2023-03-03T06:25:13.310005+0000","last_unstale":"2023-03-03T06:25:18.341996+0000","last_undegraded":"2023-03-03T06:25:18.341996+0000","last_fullsized":"2023-03-03T06:25:18.341996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217857+0000","last_change":"2023-03-03T06:25:18.362444+0000","last_active":"2023-03-03T06:25:19.217857+0000","last_peered":"2023-03-03T06:25:19.217857+0000","last_clean":"2023-03-03T06:25:19.217857+0000","last_became_active":"2023-03-03T06:25:18.362087+0000","last_became_peered":"2023-03-03T06:25:18.362087+0000","last_unstale":"2023-03-03T06:25:19.217857+0000","last_undegraded":"2023-03-03T06:25:19.217857+0000","last_fullsized":"2023-03-03T06:25:19.217857+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:52:17.597685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343862+0000","last_change":"2023-03-03T06:25:13.834113+0000","last_active":"2023-03-03T06:25:18.343862+0000","last_peered":"2023-03-03T06:25:18.343862+0000","last_clean":"2023-03-03T06:25:18.343862+0000","last_became_active":"2023-03-03T06:25:13.833874+0000","last_became_peered":"2023-03-03T06:25:13.833874+0000","last_unstale":"2023-03-03T06:25:18.343862+0000","last_undegraded":"2023-03-03T06:25:18.343862+0000","last_fullsized":"2023-03-03T06:25:18.343862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352647+0000","last_change":"2023-03-03T06:25:13.310942+0000","last_active":"2023-03-03T06:25:18.352647+0000","last_peered":"2023-03-03T06:25:18.352647+0000","last_clean":"2023-03-03T06:25:18.352647+0000","last_became_active":"2023-03-03T06:25:13.310632+0000","last_became_peered":"2023-03-03T06:25:13.310632+0000","last_unstale":"2023-03-03T06:25:18.352647+0000","last_undegraded":"2023-03-03T06:25:18.352647+0000","last_fullsized":"2023-03-03T06:25:18.352647+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:16:16.586666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217181+0000","last_change":"2023-03-03T06:25:18.360063+0000","last_active":"2023-03-03T06:25:19.217181+0000","last_peered":"2023-03-03T06:25:19.217181+0000","last_clean":"2023-03-03T06:25:19.217181+0000","last_became_active":"2023-03-03T06:25:18.359549+0000","last_became_peered":"2023-03-03T06:25:18.359549+0000","last_unstale":"2023-03-03T06:25:19.217181+0000","last_undegraded":"2023-03-03T06:25:19.217181+0000","last_fullsized":"2023-03-03T06:25:19.217181+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:30:28.010892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341203+0000","last_change":"2023-03-03T06:25:13.844840+0000","last_active":"2023-03-03T06:25:18.341203+0000","last_peered":"2023-03-03T06:25:18.341203+0000","last_clean":"2023-03-03T06:25:18.341203+0000","last_became_active":"2023-03-03T06:25:13.844556+0000","last_became_peered":"2023-03-03T06:25:13.844556+0000","last_unstale":"2023-03-03T06:25:18.341203+0000","last_undegraded":"2023-03-03T06:25:18.341203+0000","last_fullsized":"2023-03-03T06:25:18.341203+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:19:49.547542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350702+0000","last_change":"2023-03-03T06:25:13.303131+0000","last_active":"2023-03-03T06:25:18.350702+0000","last_peered":"2023-03-03T06:25:18.350702+0000","last_clean":"2023-03-03T06:25:18.350702+0000","last_became_active":"2023-03-03T06:25:13.302861+0000","last_became_peered":"2023-03-03T06:25:13.302861+0000","last_unstale":"2023-03-03T06:25:18.350702+0000","last_undegraded":"2023-03-03T06:25:18.350702+0000","last_fullsized":"2023-03-03T06:25:18.350702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:46:07.297374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217444+0000","last_change":"2023-03-03T06:25:18.363570+0000","last_active":"2023-03-03T06:25:19.217444+0000","last_peered":"2023-03-03T06:25:19.217444+0000","last_clean":"2023-03-03T06:25:19.217444+0000","last_became_active":"2023-03-03T06:25:18.363224+0000","last_became_peered":"2023-03-03T06:25:18.363224+0000","last_unstale":"2023-03-03T06:25:19.217444+0000","last_undegraded":"2023-03-03T06:25:19.217444+0000","last_fullsized":"2023-03-03T06:25:19.217444+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:38:51.150150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335118+0000","last_change":"2023-03-03T06:25:13.837508+0000","last_active":"2023-03-03T06:25:18.335118+0000","last_peered":"2023-03-03T06:25:18.335118+0000","last_clean":"2023-03-03T06:25:18.335118+0000","last_became_active":"2023-03-03T06:25:13.836864+0000","last_became_peered":"2023-03-03T06:25:13.836864+0000","last_unstale":"2023-03-03T06:25:18.335118+0000","last_undegraded":"2023-03-03T06:25:18.335118+0000","last_fullsized":"2023-03-03T06:25:18.335118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:33:19.697671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350575+0000","last_change":"2023-03-03T06:25:13.308653+0000","last_active":"2023-03-03T06:25:18.350575+0000","last_peered":"2023-03-03T06:25:18.350575+0000","last_clean":"2023-03-03T06:25:18.350575+0000","last_became_active":"2023-03-03T06:25:13.308404+0000","last_became_peered":"2023-03-03T06:25:13.308404+0000","last_unstale":"2023-03-03T06:25:18.350575+0000","last_undegraded":"2023-03-03T06:25:18.350575+0000","last_fullsized":"2023-03-03T06:25:18.350575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:42:29.976451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217056+0000","last_change":"2023-03-03T06:25:18.369545+0000","last_active":"2023-03-03T06:25:19.217056+0000","last_peered":"2023-03-03T06:25:19.217056+0000","last_clean":"2023-03-03T06:25:19.217056+0000","last_became_active":"2023-03-03T06:25:18.369075+0000","last_became_peered":"2023-03-03T06:25:18.369075+0000","last_unstale":"2023-03-03T06:25:19.217056+0000","last_undegraded":"2023-03-03T06:25:19.217056+0000","last_fullsized":"2023-03-03T06:25:19.217056+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:45:30.096214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352873+0000","last_change":"2023-03-03T06:25:13.836258+0000","last_active":"2023-03-03T06:25:18.352873+0000","last_peered":"2023-03-03T06:25:18.352873+0000","last_clean":"2023-03-03T06:25:18.352873+0000","last_became_active":"2023-03-03T06:25:13.835993+0000","last_became_peered":"2023-03-03T06:25:13.835993+0000","last_unstale":"2023-03-03T06:25:18.352873+0000","last_undegraded":"2023-03-03T06:25:18.352873+0000","last_fullsized":"2023-03-03T06:25:18.352873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350886+0000","last_change":"2023-03-03T06:25:13.307887+0000","last_active":"2023-03-03T06:25:18.350886+0000","last_peered":"2023-03-03T06:25:18.350886+0000","last_clean":"2023-03-03T06:25:18.350886+0000","last_became_active":"2023-03-03T06:25:13.307622+0000","last_became_peered":"2023-03-03T06:25:13.307622+0000","last_unstale":"2023-03-03T06:25:18.350886+0000","last_undegraded":"2023-03-03T06:25:18.350886+0000","last_fullsized":"2023-03-03T06:25:18.350886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:51:44.342844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.462959+0000","last_change":"2023-03-03T06:25:18.366887+0000","last_active":"2023-03-03T06:25:19.462959+0000","last_peered":"2023-03-03T06:25:19.462959+0000","last_clean":"2023-03-03T06:25:19.462959+0000","last_became_active":"2023-03-03T06:25:18.366248+0000","last_became_peered":"2023-03-03T06:25:18.366248+0000","last_unstale":"2023-03-03T06:25:19.462959+0000","last_undegraded":"2023-03-03T06:25:19.462959+0000","last_fullsized":"2023-03-03T06:25:19.462959+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:06:08.404601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879274+0000","last_change":"2023-03-03T06:25:18.374976+0000","last_active":"2023-03-03T06:25:18.879274+0000","last_peered":"2023-03-03T06:25:18.879274+0000","last_clean":"2023-03-03T06:25:18.879274+0000","last_became_active":"2023-03-03T06:25:18.374414+0000","last_became_peered":"2023-03-03T06:25:18.374414+0000","last_unstale":"2023-03-03T06:25:18.879274+0000","last_undegraded":"2023-03-03T06:25:18.879274+0000","last_fullsized":"2023-03-03T06:25:18.879274+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:52:08.870718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.340487+0000","last_change":"2023-03-03T06:25:13.297905+0000","last_active":"2023-03-03T06:25:18.340487+0000","last_peered":"2023-03-03T06:25:18.340487+0000","last_clean":"2023-03-03T06:25:18.340487+0000","last_became_active":"2023-03-03T06:25:13.297568+0000","last_became_peered":"2023-03-03T06:25:13.297568+0000","last_unstale":"2023-03-03T06:25:18.340487+0000","last_undegraded":"2023-03-03T06:25:18.340487+0000","last_fullsized":"2023-03-03T06:25:18.340487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:17:34.388274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343217+0000","last_change":"2023-03-03T06:25:13.851715+0000","last_active":"2023-03-03T06:25:18.343217+0000","last_peered":"2023-03-03T06:25:18.343217+0000","last_clean":"2023-03-03T06:25:18.343217+0000","last_became_active":"2023-03-03T06:25:13.851377+0000","last_became_peered":"2023-03-03T06:25:13.851377+0000","last_unstale":"2023-03-03T06:25:18.343217+0000","last_undegraded":"2023-03-03T06:25:18.343217+0000","last_fullsized":"2023-03-03T06:25:18.343217+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467810+0000","last_change":"2023-03-03T06:25:18.356507+0000","last_active":"2023-03-03T06:25:19.467810+0000","last_peered":"2023-03-03T06:25:19.467810+0000","last_clean":"2023-03-03T06:25:19.467810+0000","last_became_active":"2023-03-03T06:25:18.356097+0000","last_became_peered":"2023-03-03T06:25:18.356097+0000","last_unstale":"2023-03-03T06:25:19.467810+0000","last_undegraded":"2023-03-03T06:25:19.467810+0000","last_fullsized":"2023-03-03T06:25:19.467810+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:59:33.700378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.351468+0000","last_change":"2023-03-03T06:25:13.306049+0000","last_active":"2023-03-03T06:25:18.351468+0000","last_peered":"2023-03-03T06:25:18.351468+0000","last_clean":"2023-03-03T06:25:18.351468+0000","last_became_active":"2023-03-03T06:25:13.305742+0000","last_became_peered":"2023-03-03T06:25:13.305742+0000","last_unstale":"2023-03-03T06:25:18.351468+0000","last_undegraded":"2023-03-03T06:25:18.351468+0000","last_fullsized":"2023-03-03T06:25:18.351468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:29:05.784203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344608+0000","last_change":"2023-03-03T06:25:13.841823+0000","last_active":"2023-03-03T06:25:18.344608+0000","last_peered":"2023-03-03T06:25:18.344608+0000","last_clean":"2023-03-03T06:25:18.344608+0000","last_became_active":"2023-03-03T06:25:13.841581+0000","last_became_peered":"2023-03-03T06:25:13.841581+0000","last_unstale":"2023-03-03T06:25:18.344608+0000","last_undegraded":"2023-03-03T06:25:18.344608+0000","last_fullsized":"2023-03-03T06:25:18.344608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342620+0000","last_change":"2023-03-03T06:25:13.307869+0000","last_active":"2023-03-03T06:25:18.342620+0000","last_peered":"2023-03-03T06:25:18.342620+0000","last_clean":"2023-03-03T06:25:18.342620+0000","last_became_active":"2023-03-03T06:25:13.303371+0000","last_became_peered":"2023-03-03T06:25:13.303371+0000","last_unstale":"2023-03-03T06:25:18.342620+0000","last_undegraded":"2023-03-03T06:25:18.342620+0000","last_fullsized":"2023-03-03T06:25:18.342620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352774+0000","last_change":"2023-03-03T06:25:13.853988+0000","last_active":"2023-03-03T06:25:18.352774+0000","last_peered":"2023-03-03T06:25:18.352774+0000","last_clean":"2023-03-03T06:25:18.352774+0000","last_became_active":"2023-03-03T06:25:13.853714+0000","last_became_peered":"2023-03-03T06:25:13.853714+0000","last_unstale":"2023-03-03T06:25:18.352774+0000","last_undegraded":"2023-03-03T06:25:18.352774+0000","last_fullsized":"2023-03-03T06:25:18.352774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:51:30.984078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467763+0000","last_change":"2023-03-03T06:25:18.372232+0000","last_active":"2023-03-03T06:25:19.467763+0000","last_peered":"2023-03-03T06:25:19.467763+0000","last_clean":"2023-03-03T06:25:19.467763+0000","last_became_active":"2023-03-03T06:25:18.371809+0000","last_became_peered":"2023-03-03T06:25:18.371809+0000","last_unstale":"2023-03-03T06:25:19.467763+0000","last_undegraded":"2023-03-03T06:25:19.467763+0000","last_fullsized":"2023-03-03T06:25:19.467763+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:11:23.940316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346535+0000","last_change":"2023-03-03T06:25:13.307069+0000","last_active":"2023-03-03T06:25:18.346535+0000","last_peered":"2023-03-03T06:25:18.346535+0000","last_clean":"2023-03-03T06:25:18.346535+0000","last_became_active":"2023-03-03T06:25:13.306791+0000","last_became_peered":"2023-03-03T06:25:13.306791+0000","last_unstale":"2023-03-03T06:25:18.346535+0000","last_undegraded":"2023-03-03T06:25:18.346535+0000","last_fullsized":"2023-03-03T06:25:18.346535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:19:48.531435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343519+0000","last_change":"2023-03-03T06:25:13.830026+0000","last_active":"2023-03-03T06:25:18.343519+0000","last_peered":"2023-03-03T06:25:18.343519+0000","last_clean":"2023-03-03T06:25:18.343519+0000","last_became_active":"2023-03-03T06:25:13.829773+0000","last_became_peered":"2023-03-03T06:25:13.829773+0000","last_unstale":"2023-03-03T06:25:18.343519+0000","last_undegraded":"2023-03-03T06:25:18.343519+0000","last_fullsized":"2023-03-03T06:25:18.343519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:35:57.127264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879167+0000","last_change":"2023-03-03T06:25:18.374930+0000","last_active":"2023-03-03T06:25:18.879167+0000","last_peered":"2023-03-03T06:25:18.879167+0000","last_clean":"2023-03-03T06:25:18.879167+0000","last_became_active":"2023-03-03T06:25:18.374374+0000","last_became_peered":"2023-03-03T06:25:18.374374+0000","last_unstale":"2023-03-03T06:25:18.879167+0000","last_undegraded":"2023-03-03T06:25:18.879167+0000","last_fullsized":"2023-03-03T06:25:18.879167+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:09:37.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.340359+0000","last_change":"2023-03-03T06:25:13.840311+0000","last_active":"2023-03-03T06:25:18.340359+0000","last_peered":"2023-03-03T06:25:18.340359+0000","last_clean":"2023-03-03T06:25:18.340359+0000","last_became_active":"2023-03-03T06:25:13.839648+0000","last_became_peered":"2023-03-03T06:25:13.839648+0000","last_unstale":"2023-03-03T06:25:18.340359+0000","last_undegraded":"2023-03-03T06:25:18.340359+0000","last_fullsized":"2023-03-03T06:25:18.340359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:29:06.800310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337721+0000","last_change":"2023-03-03T06:25:13.836213+0000","last_active":"2023-03-03T06:25:18.337721+0000","last_peered":"2023-03-03T06:25:18.337721+0000","last_clean":"2023-03-03T06:25:18.337721+0000","last_became_active":"2023-03-03T06:25:13.835890+0000","last_became_peered":"2023-03-03T06:25:13.835890+0000","last_unstale":"2023-03-03T06:25:18.337721+0000","last_undegraded":"2023-03-03T06:25:18.337721+0000","last_fullsized":"2023-03-03T06:25:18.337721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:55.025383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335018+0000","last_change":"2023-03-03T06:25:13.834786+0000","last_active":"2023-03-03T06:25:18.335018+0000","last_peered":"2023-03-03T06:25:18.335018+0000","last_clean":"2023-03-03T06:25:18.335018+0000","last_became_active":"2023-03-03T06:25:13.834531+0000","last_became_peered":"2023-03-03T06:25:13.834531+0000","last_unstale":"2023-03-03T06:25:18.335018+0000","last_undegraded":"2023-03-03T06:25:18.335018+0000","last_fullsized":"2023-03-03T06:25:18.335018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:07:17.177238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342130+0000","last_change":"2023-03-03T06:25:13.850334+0000","last_active":"2023-03-03T06:25:18.342130+0000","last_peered":"2023-03-03T06:25:18.342130+0000","last_clean":"2023-03-03T06:25:18.342130+0000","last_became_active":"2023-03-03T06:25:13.850082+0000","last_became_peered":"2023-03-03T06:25:13.850082+0000","last_unstale":"2023-03-03T06:25:18.342130+0000","last_undegraded":"2023-03-03T06:25:18.342130+0000","last_fullsized":"2023-03-03T06:25:18.342130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357706+0000","last_change":"2023-03-03T06:25:13.850690+0000","last_active":"2023-03-03T06:25:18.357706+0000","last_peered":"2023-03-03T06:25:18.357706+0000","last_clean":"2023-03-03T06:25:18.357706+0000","last_became_active":"2023-03-03T06:25:13.850392+0000","last_became_peered":"2023-03-03T06:25:13.850392+0000","last_unstale":"2023-03-03T06:25:18.357706+0000","last_undegraded":"2023-03-03T06:25:18.357706+0000","last_fullsized":"2023-03-03T06:25:18.357706+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352673+0000","last_change":"2023-03-03T06:25:13.853965+0000","last_active":"2023-03-03T06:25:18.352673+0000","last_peered":"2023-03-03T06:25:18.352673+0000","last_clean":"2023-03-03T06:25:18.352673+0000","last_became_active":"2023-03-03T06:25:13.853689+0000","last_became_peered":"2023-03-03T06:25:13.853689+0000","last_unstale":"2023-03-03T06:25:18.352673+0000","last_undegraded":"2023-03-03T06:25:18.352673+0000","last_fullsized":"2023-03-03T06:25:18.352673+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:09:17.786520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357579+0000","last_change":"2023-03-03T06:25:13.828945+0000","last_active":"2023-03-03T06:25:18.357579+0000","last_peered":"2023-03-03T06:25:18.357579+0000","last_clean":"2023-03-03T06:25:18.357579+0000","last_became_active":"2023-03-03T06:25:13.828655+0000","last_became_peered":"2023-03-03T06:25:13.828655+0000","last_unstale":"2023-03-03T06:25:18.357579+0000","last_undegraded":"2023-03-03T06:25:18.357579+0000","last_fullsized":"2023-03-03T06:25:18.357579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:51:30.984078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358304+0000","last_change":"2023-03-03T06:25:13.834307+0000","last_active":"2023-03-03T06:25:18.358304+0000","last_peered":"2023-03-03T06:25:18.358304+0000","last_clean":"2023-03-03T06:25:18.358304+0000","last_became_active":"2023-03-03T06:25:13.834054+0000","last_became_peered":"2023-03-03T06:25:13.834054+0000","last_unstale":"2023-03-03T06:25:18.358304+0000","last_undegraded":"2023-03-03T06:25:18.358304+0000","last_fullsized":"2023-03-03T06:25:18.358304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:54:02.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346796+0000","last_change":"2023-03-03T06:25:13.856830+0000","last_active":"2023-03-03T06:25:18.346796+0000","last_peered":"2023-03-03T06:25:18.346796+0000","last_clean":"2023-03-03T06:25:18.346796+0000","last_became_active":"2023-03-03T06:25:13.856494+0000","last_became_peered":"2023-03-03T06:25:13.856494+0000","last_unstale":"2023-03-03T06:25:18.346796+0000","last_undegraded":"2023-03-03T06:25:18.346796+0000","last_fullsized":"2023-03-03T06:25:18.346796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:59:50.232210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337609+0000","last_change":"2023-03-03T06:25:13.836186+0000","last_active":"2023-03-03T06:25:18.337609+0000","last_peered":"2023-03-03T06:25:18.337609+0000","last_clean":"2023-03-03T06:25:18.337609+0000","last_became_active":"2023-03-03T06:25:13.835481+0000","last_became_peered":"2023-03-03T06:25:13.835481+0000","last_unstale":"2023-03-03T06:25:18.337609+0000","last_undegraded":"2023-03-03T06:25:18.337609+0000","last_fullsized":"2023-03-03T06:25:18.337609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:30:23.964335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342530+0000","last_change":"2023-03-03T06:25:13.834687+0000","last_active":"2023-03-03T06:25:18.342530+0000","last_peered":"2023-03-03T06:25:18.342530+0000","last_clean":"2023-03-03T06:25:18.342530+0000","last_became_active":"2023-03-03T06:25:13.834424+0000","last_became_peered":"2023-03-03T06:25:13.834424+0000","last_unstale":"2023-03-03T06:25:18.342530+0000","last_undegraded":"2023-03-03T06:25:18.342530+0000","last_fullsized":"2023-03-03T06:25:18.342530+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346438+0000","last_change":"2023-03-03T06:25:13.855849+0000","last_active":"2023-03-03T06:25:18.346438+0000","last_peered":"2023-03-03T06:25:18.346438+0000","last_clean":"2023-03-03T06:25:18.346438+0000","last_became_active":"2023-03-03T06:25:13.855609+0000","last_became_peered":"2023-03-03T06:25:13.855609+0000","last_unstale":"2023-03-03T06:25:18.346438+0000","last_undegraded":"2023-03-03T06:25:18.346438+0000","last_fullsized":"2023-03-03T06:25:18.346438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335348+0000","last_change":"2023-03-03T06:25:13.846670+0000","last_active":"2023-03-03T06:25:18.335348+0000","last_peered":"2023-03-03T06:25:18.335348+0000","last_clean":"2023-03-03T06:25:18.335348+0000","last_became_active":"2023-03-03T06:25:13.846349+0000","last_became_peered":"2023-03-03T06:25:13.846349+0000","last_unstale":"2023-03-03T06:25:18.335348+0000","last_undegraded":"2023-03-03T06:25:18.335348+0000","last_fullsized":"2023-03-03T06:25:18.335348+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:30:23.964335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337310+0000","last_change":"2023-03-03T06:25:13.839483+0000","last_active":"2023-03-03T06:25:18.337310+0000","last_peered":"2023-03-03T06:25:18.337310+0000","last_clean":"2023-03-03T06:25:18.337310+0000","last_became_active":"2023-03-03T06:25:13.838497+0000","last_became_peered":"2023-03-03T06:25:13.838497+0000","last_unstale":"2023-03-03T06:25:18.337310+0000","last_undegraded":"2023-03-03T06:25:18.337310+0000","last_fullsized":"2023-03-03T06:25:18.337310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:08.122967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334979+0000","last_change":"2023-03-03T06:25:13.855262+0000","last_active":"2023-03-03T06:25:18.334979+0000","last_peered":"2023-03-03T06:25:18.334979+0000","last_clean":"2023-03-03T06:25:18.334979+0000","last_became_active":"2023-03-03T06:25:13.848660+0000","last_became_peered":"2023-03-03T06:25:13.848660+0000","last_unstale":"2023-03-03T06:25:18.334979+0000","last_undegraded":"2023-03-03T06:25:18.334979+0000","last_fullsized":"2023-03-03T06:25:18.334979+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:55.025383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344570+0000","last_change":"2023-03-03T06:25:13.842845+0000","last_active":"2023-03-03T06:25:18.344570+0000","last_peered":"2023-03-03T06:25:18.344570+0000","last_clean":"2023-03-03T06:25:18.344570+0000","last_became_active":"2023-03-03T06:25:13.842580+0000","last_became_peered":"2023-03-03T06:25:13.842580+0000","last_unstale":"2023-03-03T06:25:18.344570+0000","last_undegraded":"2023-03-03T06:25:18.344570+0000","last_fullsized":"2023-03-03T06:25:18.344570+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:28:42.514217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335348+0000","last_change":"2023-03-03T06:25:13.836306+0000","last_active":"2023-03-03T06:25:18.335348+0000","last_peered":"2023-03-03T06:25:18.335348+0000","last_clean":"2023-03-03T06:25:18.335348+0000","last_became_active":"2023-03-03T06:25:13.835953+0000","last_became_peered":"2023-03-03T06:25:13.835953+0000","last_unstale":"2023-03-03T06:25:18.335348+0000","last_undegraded":"2023-03-03T06:25:18.335348+0000","last_fullsized":"2023-03-03T06:25:18.335348+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:35:57.127264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.351349+0000","last_change":"2023-03-03T06:25:13.854132+0000","last_active":"2023-03-03T06:25:18.351349+0000","last_peered":"2023-03-03T06:25:18.351349+0000","last_clean":"2023-03-03T06:25:18.351349+0000","last_became_active":"2023-03-03T06:25:13.853786+0000","last_became_peered":"2023-03-03T06:25:13.853786+0000","last_unstale":"2023-03-03T06:25:18.351349+0000","last_undegraded":"2023-03-03T06:25:18.351349+0000","last_fullsized":"2023-03-03T06:25:18.351349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353001+0000","last_change":"2023-03-03T06:25:13.835134+0000","last_active":"2023-03-03T06:25:18.353001+0000","last_peered":"2023-03-03T06:25:18.353001+0000","last_clean":"2023-03-03T06:25:18.353001+0000","last_became_active":"2023-03-03T06:25:13.834844+0000","last_became_peered":"2023-03-03T06:25:13.834844+0000","last_unstale":"2023-03-03T06:25:18.353001+0000","last_undegraded":"2023-03-03T06:25:18.353001+0000","last_fullsized":"2023-03-03T06:25:18.353001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337727+0000","last_change":"2023-03-03T06:25:13.830031+0000","last_active":"2023-03-03T06:25:18.337727+0000","last_peered":"2023-03-03T06:25:18.337727+0000","last_clean":"2023-03-03T06:25:18.337727+0000","last_became_active":"2023-03-03T06:25:13.829770+0000","last_became_peered":"2023-03-03T06:25:13.829770+0000","last_unstale":"2023-03-03T06:25:18.337727+0000","last_undegraded":"2023-03-03T06:25:18.337727+0000","last_fullsized":"2023-03-03T06:25:18.337727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:06.900541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.336311+0000","last_change":"2023-03-03T06:25:13.858248+0000","last_active":"2023-03-03T06:25:18.336311+0000","last_peered":"2023-03-03T06:25:18.336311+0000","last_clean":"2023-03-03T06:25:18.336311+0000","last_became_active":"2023-03-03T06:25:13.857884+0000","last_became_peered":"2023-03-03T06:25:13.857884+0000","last_unstale":"2023-03-03T06:25:18.336311+0000","last_undegraded":"2023-03-03T06:25:18.336311+0000","last_fullsized":"2023-03-03T06:25:18.336311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:39.051115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334811+0000","last_change":"2023-03-03T06:25:13.847616+0000","last_active":"2023-03-03T06:25:18.334811+0000","last_peered":"2023-03-03T06:25:18.334811+0000","last_clean":"2023-03-03T06:25:18.334811+0000","last_became_active":"2023-03-03T06:25:13.847110+0000","last_became_peered":"2023-03-03T06:25:13.847110+0000","last_unstale":"2023-03-03T06:25:18.334811+0000","last_undegraded":"2023-03-03T06:25:18.334811+0000","last_fullsized":"2023-03-03T06:25:18.334811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:52:13.551129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353150+0000","last_change":"2023-03-03T06:25:13.851731+0000","last_active":"2023-03-03T06:25:18.353150+0000","last_peered":"2023-03-03T06:25:18.353150+0000","last_clean":"2023-03-03T06:25:18.353150+0000","last_became_active":"2023-03-03T06:25:13.849868+0000","last_became_peered":"2023-03-03T06:25:13.849868+0000","last_unstale":"2023-03-03T06:25:18.353150+0000","last_undegraded":"2023-03-03T06:25:18.353150+0000","last_fullsized":"2023-03-03T06:25:18.353150+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343321+0000","last_change":"2023-03-03T06:25:13.846247+0000","last_active":"2023-03-03T06:25:18.343321+0000","last_peered":"2023-03-03T06:25:18.343321+0000","last_clean":"2023-03-03T06:25:18.343321+0000","last_became_active":"2023-03-03T06:25:13.845983+0000","last_became_peered":"2023-03-03T06:25:13.845983+0000","last_unstale":"2023-03-03T06:25:18.343321+0000","last_undegraded":"2023-03-03T06:25:18.343321+0000","last_fullsized":"2023-03-03T06:25:18.343321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:54:02.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352766+0000","last_change":"2023-03-03T06:25:13.858013+0000","last_active":"2023-03-03T06:25:18.352766+0000","last_peered":"2023-03-03T06:25:18.352766+0000","last_clean":"2023-03-03T06:25:18.352766+0000","last_became_active":"2023-03-03T06:25:13.857706+0000","last_became_peered":"2023-03-03T06:25:13.857706+0000","last_unstale":"2023-03-03T06:25:18.352766+0000","last_undegraded":"2023-03-03T06:25:18.352766+0000","last_fullsized":"2023-03-03T06:25:18.352766+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:54:02.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.351182+0000","last_change":"2023-03-03T06:25:13.854691+0000","last_active":"2023-03-03T06:25:18.351182+0000","last_peered":"2023-03-03T06:25:18.351182+0000","last_clean":"2023-03-03T06:25:18.351182+0000","last_became_active":"2023-03-03T06:25:13.854332+0000","last_became_peered":"2023-03-03T06:25:13.854332+0000","last_unstale":"2023-03-03T06:25:18.351182+0000","last_undegraded":"2023-03-03T06:25:18.351182+0000","last_fullsized":"2023-03-03T06:25:18.351182+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334771+0000","last_change":"2023-03-03T06:25:13.836791+0000","last_active":"2023-03-03T06:25:18.334771+0000","last_peered":"2023-03-03T06:25:18.334771+0000","last_clean":"2023-03-03T06:25:18.334771+0000","last_became_active":"2023-03-03T06:25:13.836500+0000","last_became_peered":"2023-03-03T06:25:13.836500+0000","last_unstale":"2023-03-03T06:25:18.334771+0000","last_undegraded":"2023-03-03T06:25:18.334771+0000","last_fullsized":"2023-03-03T06:25:18.334771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:38:47.103593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344030+0000","last_change":"2023-03-03T06:25:13.846407+0000","last_active":"2023-03-03T06:25:18.344030+0000","last_peered":"2023-03-03T06:25:18.344030+0000","last_clean":"2023-03-03T06:25:18.344030+0000","last_became_active":"2023-03-03T06:25:13.846119+0000","last_became_peered":"2023-03-03T06:25:13.846119+0000","last_unstale":"2023-03-03T06:25:18.344030+0000","last_undegraded":"2023-03-03T06:25:18.344030+0000","last_fullsized":"2023-03-03T06:25:18.344030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341377+0000","last_change":"2023-03-03T06:25:13.843673+0000","last_active":"2023-03-03T06:25:18.341377+0000","last_peered":"2023-03-03T06:25:18.341377+0000","last_clean":"2023-03-03T06:25:18.341377+0000","last_became_active":"2023-03-03T06:25:13.843408+0000","last_became_peered":"2023-03-03T06:25:13.843408+0000","last_unstale":"2023-03-03T06:25:18.341377+0000","last_undegraded":"2023-03-03T06:25:18.341377+0000","last_fullsized":"2023-03-03T06:25:18.341377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:51:45.358951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335571+0000","last_change":"2023-03-03T06:25:13.847791+0000","last_active":"2023-03-03T06:25:18.335571+0000","last_peered":"2023-03-03T06:25:18.335571+0000","last_clean":"2023-03-03T06:25:18.335571+0000","last_became_active":"2023-03-03T06:25:13.847378+0000","last_became_peered":"2023-03-03T06:25:13.847378+0000","last_unstale":"2023-03-03T06:25:18.335571+0000","last_undegraded":"2023-03-03T06:25:18.335571+0000","last_fullsized":"2023-03-03T06:25:18.335571+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:05:53.754239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353098+0000","last_change":"2023-03-03T06:25:13.835727+0000","last_active":"2023-03-03T06:25:18.353098+0000","last_peered":"2023-03-03T06:25:18.353098+0000","last_clean":"2023-03-03T06:25:18.353098+0000","last_became_active":"2023-03-03T06:25:13.835462+0000","last_became_peered":"2023-03-03T06:25:13.835462+0000","last_unstale":"2023-03-03T06:25:18.353098+0000","last_undegraded":"2023-03-03T06:25:18.353098+0000","last_fullsized":"2023-03-03T06:25:18.353098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.336596+0000","last_change":"2023-03-03T06:25:13.827811+0000","last_active":"2023-03-03T06:25:18.336596+0000","last_peered":"2023-03-03T06:25:18.336596+0000","last_clean":"2023-03-03T06:25:18.336596+0000","last_became_active":"2023-03-03T06:25:13.827532+0000","last_became_peered":"2023-03-03T06:25:13.827532+0000","last_unstale":"2023-03-03T06:25:18.336596+0000","last_undegraded":"2023-03-03T06:25:18.336596+0000","last_fullsized":"2023-03-03T06:25:18.336596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:09.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4527,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13456,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":12544,"kb_avail":94358384,"statfs":{"total":96636764160,"available":96622985216,"internally_reserved":0,"allocated":925696,"data_stored":718901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12845056},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.508}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6439999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12272,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":11968,"kb_avail":94359568,"statfs":{"total":96636764160,"available":96624197632,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12255232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7070000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6279999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6040000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12280,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":11968,"kb_avail":94359560,"statfs":{"total":96636764160,"available":96624189440,"internally_reserved":0,"allocated":311296,"data_stored":125208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12255232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6040000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.544}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6739999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12356,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":12032,"kb_avail":94359484,"statfs":{"total":96636764160,"available":96624111616,"internally_reserved":0,"allocated":323584,"data_stored":128373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12320768},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4870000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.843}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7749999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2080000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1520000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8112,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363728,"statfs":{"total":96636764160,"available":96628457472,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8240,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363600,"statfs":{"total":96636764160,"available":96628326400,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9272,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362568,"statfs":{"total":96636764160,"available":96627269632,"internally_reserved":0,"allocated":901120,"data_stored":715506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8620000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7050000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.758}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-03T06:25:48.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-03T06:25:48.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.971+0000 7f2185343700 1 Processor -- start 2023-03-03T06:25:48.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.971+0000 7f2185343700 1 -- start start 2023-03-03T06:25:48.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.971+0000 7f2185343700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f218006a780 0x7f2180158440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.971+0000 7f2185343700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21800aee70 0x7f218015c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.971+0000 7f2185343700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800ca2a0 con 0x7f218015c580 2023-03-03T06:25:48.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.971+0000 7f2185343700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218015ca20 con 0x7f218006a780 2023-03-03T06:25:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.971+0000 7f2185343700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21800cdd30 con 0x7f21800aee70 2023-03-03T06:25:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.971+0000 7f2184b42700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f218015c580 0x7f21800cb010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33788/0 (socket says 172.21.15.3:33788) 2023-03-03T06:25:48.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.971+0000 7f2184b42700 1 -- 172.21.15.3:0/3038174512 learned_addr learned my addr 172.21.15.3:0/3038174512 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:48.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.971+0000 7f217f7fe700 1 --2- 172.21.15.3:0/3038174512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21800aee70 0x7f218015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:48.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.971+0000 7f217ffff700 1 --2- 172.21.15.3:0/3038174512 >> v2:172.21.15.32:3300/0 conn(0x7f218006a780 0x7f2180158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:48.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.975+0000 7f217effd700 1 -- 172.21.15.3:0/3038174512 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582952954 0 0) 0x7f21800ca2a0 con 0x7f218015c580 2023-03-03T06:25:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.975+0000 7f217effd700 1 -- 172.21.15.3:0/3038174512 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21680035d0 con 0x7f218015c580 2023-03-03T06:25:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.975+0000 7f217f7fe700 1 -- 172.21.15.3:0/3038174512 >> v2:172.21.15.32:3300/0 conn(0x7f218006a780 msgr2=0x7f2180158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.975+0000 7f217f7fe700 1 --2- 172.21.15.3:0/3038174512 >> v2:172.21.15.32:3300/0 conn(0x7f218006a780 0x7f2180158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.975+0000 7f217f7fe700 1 -- 172.21.15.3:0/3038174512 >> v1:172.21.15.3:6789/0 conn(0x7f218015c580 legacy=0x7f21800cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.975+0000 7f217f7fe700 1 -- 172.21.15.3:0/3038174512 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2180158980 con 0x7f21800aee70 2023-03-03T06:25:48.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.975+0000 7f217ffff700 1 --2- 172.21.15.3:0/3038174512 >> v2:172.21.15.32:3300/0 conn(0x7f218006a780 0x7f2180158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:48.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.975+0000 7f217f7fe700 1 --2- 172.21.15.3:0/3038174512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21800aee70 0x7f218015c040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2174009850 tx=0x7f2174009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52ae6f83a97de1eb server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:48.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.975+0000 7f217effd700 1 -- 172.21.15.3:0/3038174512 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174011830 con 0x7f21800aee70 2023-03-03T06:25:48.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.975+0000 7f217effd700 1 -- 172.21.15.3:0/3038174512 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2174011990 con 0x7f21800aee70 2023-03-03T06:25:48.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.975+0000 7f217effd700 1 -- 172.21.15.3:0/3038174512 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174011d40 con 0x7f21800aee70 2023-03-03T06:25:48.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.975+0000 7f2185343700 1 -- 172.21.15.3:0/3038174512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21800aee70 msgr2=0x7f218015c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.975+0000 7f2185343700 1 --2- 172.21.15.3:0/3038174512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21800aee70 0x7f218015c040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2174009850 tx=0x7f2174009b60 comp rx=0 tx=0).stop 2023-03-03T06:25:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.979+0000 7f2185343700 1 -- 172.21.15.3:0/3038174512 shutdown_connections 2023-03-03T06:25:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.979+0000 7f2185343700 1 --2- 172.21.15.3:0/3038174512 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21800aee70 0x7f218015c040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.979+0000 7f2185343700 1 --2- 172.21.15.3:0/3038174512 >> v2:172.21.15.32:3300/0 conn(0x7f218006a780 0x7f2180158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.979+0000 7f2185343700 1 -- 172.21.15.3:0/3038174512 >> 172.21.15.3:0/3038174512 conn(0x7f2180070a30 msgr2=0x7f2180070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.979+0000 7f2185343700 1 -- 172.21.15.3:0/3038174512 shutdown_connections 2023-03-03T06:25:48.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.979+0000 7f2185343700 1 -- 172.21.15.3:0/3038174512 wait complete. 2023-03-03T06:25:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.979+0000 7f2185343700 1 Processor -- start 2023-03-03T06:25:48.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.979+0000 7f2185343700 1 -- start start 2023-03-03T06:25:48.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2185343700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f218006a780 0x7f218014c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2185343700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21800aee70 0x7f21800c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2185343700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218015c580 0x7f2180156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2185343700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21800b2ae0 con 0x7f218015c580 2023-03-03T06:25:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2185343700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180159560 con 0x7f218006a780 2023-03-03T06:25:48.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2185343700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2180159840 con 0x7f21800aee70 2023-03-03T06:25:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2184b42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218015c580 0x7f2180156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2184b42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218015c580 0x7f2180156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43746/0 (socket says 172.21.15.3:43746) 2023-03-03T06:25:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2184b42700 1 -- 172.21.15.3:0/3726443743 learned_addr learned my addr 172.21.15.3:0/3726443743 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f217f7fe700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21800aee70 0x7f21800c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:48.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f217ffff700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f218006a780 0x7f218014c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2184b42700 1 -- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21800aee70 msgr2=0x7f21800c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2184b42700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21800aee70 0x7f21800c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2184b42700 1 -- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f218006a780 msgr2=0x7f218014c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2184b42700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f218006a780 0x7f218014c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2184b42700 1 -- 172.21.15.3:0/3726443743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2170009a70 con 0x7f218015c580 2023-03-03T06:25:48.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2184b42700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218015c580 0x7f2180156e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f217800c4d0 tx=0x7f217800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:48.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f217cff9700 1 -- 172.21.15.3:0/3726443743 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178005d90 con 0x7f218015c580 2023-03-03T06:25:48.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f217cff9700 1 -- 172.21.15.3:0/3726443743 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21780070d0 con 0x7f218015c580 2023-03-03T06:25:48.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2185343700 1 -- 172.21.15.3:0/3726443743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21740094b0 con 0x7f218015c580 2023-03-03T06:25:48.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f217cff9700 1 -- 172.21.15.3:0/3726443743 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178007460 con 0x7f218015c580 2023-03-03T06:25:48.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.983+0000 7f2185343700 1 -- 172.21.15.3:0/3726443743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21800cd6e0 con 0x7f218015c580 2023-03-03T06:25:48.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.987+0000 7f2185343700 1 -- 172.21.15.3:0/3726443743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21800ca4d0 con 0x7f218015c580 2023-03-03T06:25:48.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.987+0000 7f217cff9700 1 -- 172.21.15.3:0/3726443743 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f2178019030 con 0x7f218015c580 2023-03-03T06:25:48.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.991+0000 7f217cff9700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f216c064ee0 0x7f216c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.991+0000 7f217cff9700 1 -- 172.21.15.3:0/3726443743 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2178081590 con 0x7f218015c580 2023-03-03T06:25:48.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.991+0000 7f217ffff700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f216c064ee0 0x7f216c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:48.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.991+0000 7f217ffff700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f216c064ee0 0x7f216c067390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2170007600 tx=0x7f217000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:49.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:48.995+0000 7f217cff9700 1 -- 172.21.15.3:0/3726443743 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f217804f350 con 0x7f218015c580 2023-03-03T06:25:49.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.159+0000 7f2185343700 1 -- 172.21.15.3:0/3726443743 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21800cd9c0 con 0x7f216c064ee0 2023-03-03T06:25:49.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.179+0000 7f217cff9700 1 -- 172.21.15.3:0/3726443743 <== mgr.4105 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367170 (secure 0 0 0) 0x7f21800cd9c0 con 0x7f216c064ee0 2023-03-03T06:25:49.186 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.191+0000 7f2185343700 1 -- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f216c064ee0 msgr2=0x7f216c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.191+0000 7f2185343700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f216c064ee0 0x7f216c067390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2170007600 tx=0x7f217000b040 comp rx=0 tx=0).stop 2023-03-03T06:25:49.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.191+0000 7f2185343700 1 -- 172.21.15.3:0/3726443743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218015c580 msgr2=0x7f2180156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.191+0000 7f2185343700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218015c580 0x7f2180156e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f217800c4d0 tx=0x7f217800c7e0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.191+0000 7f2185343700 1 -- 172.21.15.3:0/3726443743 shutdown_connections 2023-03-03T06:25:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.191+0000 7f2185343700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f216c064ee0 0x7f216c067390 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.191+0000 7f2185343700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218015c580 0x7f2180156e10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.191+0000 7f2185343700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21800aee70 0x7f21800c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.191+0000 7f2185343700 1 --2- 172.21.15.3:0/3726443743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f218006a780 0x7f218014c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.191+0000 7f2185343700 1 -- 172.21.15.3:0/3726443743 >> 172.21.15.3:0/3726443743 conn(0x7f2180070a30 msgr2=0x7f218006c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:49.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.191+0000 7f2185343700 1 -- 172.21.15.3:0/3726443743 shutdown_connections 2023-03-03T06:25:49.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.191+0000 7f2185343700 1 -- 172.21.15.3:0/3726443743 wait complete. 2023-03-03T06:25:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-03-03T06:25:49.233 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2023-03-03T06:25:48.410717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594895,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":84232,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":80576,"kb_avail":754890488,"statfs":{"total":773094113280,"available":773007859712,"internally_reserved":0,"allocated":3678208,"data_stored":2186366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":82509824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005690"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461806+0000","last_change":"2023-03-03T06:25:18.375133+0000","last_active":"2023-03-03T06:25:19.461806+0000","last_peered":"2023-03-03T06:25:19.461806+0000","last_clean":"2023-03-03T06:25:19.461806+0000","last_became_active":"2023-03-03T06:25:18.374500+0000","last_became_peered":"2023-03-03T06:25:18.374500+0000","last_unstale":"2023-03-03T06:25:19.461806+0000","last_undegraded":"2023-03-03T06:25:19.461806+0000","last_fullsized":"2023-03-03T06:25:19.461806+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:30.336080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343444+0000","last_change":"2023-03-03T06:25:13.301166+0000","last_active":"2023-03-03T06:25:18.343444+0000","last_peered":"2023-03-03T06:25:18.343444+0000","last_clean":"2023-03-03T06:25:18.343444+0000","last_became_active":"2023-03-03T06:25:13.300749+0000","last_became_peered":"2023-03-03T06:25:13.300749+0000","last_unstale":"2023-03-03T06:25:18.343444+0000","last_undegraded":"2023-03-03T06:25:18.343444+0000","last_fullsized":"2023-03-03T06:25:18.343444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335467+0000","last_change":"2023-03-03T06:25:13.839287+0000","last_active":"2023-03-03T06:25:18.335467+0000","last_peered":"2023-03-03T06:25:18.335467+0000","last_clean":"2023-03-03T06:25:18.335467+0000","last_became_active":"2023-03-03T06:25:13.838935+0000","last_became_peered":"2023-03-03T06:25:13.838935+0000","last_unstale":"2023-03-03T06:25:18.335467+0000","last_undegraded":"2023-03-03T06:25:18.335467+0000","last_fullsized":"2023-03-03T06:25:18.335467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:48:01.152785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.876352+0000","last_change":"2023-03-03T06:25:18.362828+0000","last_active":"2023-03-03T06:25:18.876352+0000","last_peered":"2023-03-03T06:25:18.876352+0000","last_clean":"2023-03-03T06:25:18.876352+0000","last_became_active":"2023-03-03T06:25:18.362423+0000","last_became_peered":"2023-03-03T06:25:18.362423+0000","last_unstale":"2023-03-03T06:25:18.876352+0000","last_undegraded":"2023-03-03T06:25:18.876352+0000","last_fullsized":"2023-03-03T06:25:18.876352+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:04:44.947509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.339522+0000","last_change":"2023-03-03T06:25:13.307540+0000","last_active":"2023-03-03T06:25:18.339522+0000","last_peered":"2023-03-03T06:25:18.339522+0000","last_clean":"2023-03-03T06:25:18.339522+0000","last_became_active":"2023-03-03T06:25:13.307229+0000","last_became_peered":"2023-03-03T06:25:13.307229+0000","last_unstale":"2023-03-03T06:25:18.339522+0000","last_undegraded":"2023-03-03T06:25:18.339522+0000","last_fullsized":"2023-03-03T06:25:18.339522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:09:34.703888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337997+0000","last_change":"2023-03-03T06:25:13.832118+0000","last_active":"2023-03-03T06:25:18.337997+0000","last_peered":"2023-03-03T06:25:18.337997+0000","last_clean":"2023-03-03T06:25:18.337997+0000","last_became_active":"2023-03-03T06:25:13.831744+0000","last_became_peered":"2023-03-03T06:25:13.831744+0000","last_unstale":"2023-03-03T06:25:18.337997+0000","last_undegraded":"2023-03-03T06:25:18.337997+0000","last_fullsized":"2023-03-03T06:25:18.337997+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.878930+0000","last_change":"2023-03-03T06:25:18.367879+0000","last_active":"2023-03-03T06:25:18.878930+0000","last_peered":"2023-03-03T06:25:18.878930+0000","last_clean":"2023-03-03T06:25:18.878930+0000","last_became_active":"2023-03-03T06:25:18.367232+0000","last_became_peered":"2023-03-03T06:25:18.367232+0000","last_unstale":"2023-03-03T06:25:18.878930+0000","last_undegraded":"2023-03-03T06:25:18.878930+0000","last_fullsized":"2023-03-03T06:25:18.878930+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:43:20.174095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352877+0000","last_change":"2023-03-03T06:25:13.301087+0000","last_active":"2023-03-03T06:25:18.352877+0000","last_peered":"2023-03-03T06:25:18.352877+0000","last_clean":"2023-03-03T06:25:18.352877+0000","last_became_active":"2023-03-03T06:25:13.300837+0000","last_became_peered":"2023-03-03T06:25:13.300837+0000","last_unstale":"2023-03-03T06:25:18.352877+0000","last_undegraded":"2023-03-03T06:25:18.352877+0000","last_fullsized":"2023-03-03T06:25:18.352877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334901+0000","last_change":"2023-03-03T06:25:13.830954+0000","last_active":"2023-03-03T06:25:18.334901+0000","last_peered":"2023-03-03T06:25:18.334901+0000","last_clean":"2023-03-03T06:25:18.334901+0000","last_became_active":"2023-03-03T06:25:13.830614+0000","last_became_peered":"2023-03-03T06:25:13.830614+0000","last_unstale":"2023-03-03T06:25:18.334901+0000","last_undegraded":"2023-03-03T06:25:18.334901+0000","last_fullsized":"2023-03-03T06:25:18.334901+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.463051+0000","last_change":"2023-03-03T06:25:18.366781+0000","last_active":"2023-03-03T06:25:19.463051+0000","last_peered":"2023-03-03T06:25:19.463051+0000","last_clean":"2023-03-03T06:25:19.463051+0000","last_became_active":"2023-03-03T06:25:18.366175+0000","last_became_peered":"2023-03-03T06:25:18.366175+0000","last_unstale":"2023-03-03T06:25:19.463051+0000","last_undegraded":"2023-03-03T06:25:19.463051+0000","last_fullsized":"2023-03-03T06:25:19.463051+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:30.336080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352547+0000","last_change":"2023-03-03T06:25:13.300495+0000","last_active":"2023-03-03T06:25:18.352547+0000","last_peered":"2023-03-03T06:25:18.352547+0000","last_clean":"2023-03-03T06:25:18.352547+0000","last_became_active":"2023-03-03T06:25:13.300180+0000","last_became_peered":"2023-03-03T06:25:13.300180+0000","last_unstale":"2023-03-03T06:25:18.352547+0000","last_undegraded":"2023-03-03T06:25:18.352547+0000","last_fullsized":"2023-03-03T06:25:18.352547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335053+0000","last_change":"2023-03-03T06:25:13.830988+0000","last_active":"2023-03-03T06:25:18.335053+0000","last_peered":"2023-03-03T06:25:18.335053+0000","last_clean":"2023-03-03T06:25:18.335053+0000","last_became_active":"2023-03-03T06:25:13.826124+0000","last_became_peered":"2023-03-03T06:25:13.826124+0000","last_unstale":"2023-03-03T06:25:18.335053+0000","last_undegraded":"2023-03-03T06:25:18.335053+0000","last_fullsized":"2023-03-03T06:25:18.335053+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461936+0000","last_change":"2023-03-03T06:25:18.372519+0000","last_active":"2023-03-03T06:25:19.461936+0000","last_peered":"2023-03-03T06:25:19.461936+0000","last_clean":"2023-03-03T06:25:19.461936+0000","last_became_active":"2023-03-03T06:25:18.372087+0000","last_became_peered":"2023-03-03T06:25:18.372087+0000","last_unstale":"2023-03-03T06:25:19.461936+0000","last_undegraded":"2023-03-03T06:25:19.461936+0000","last_fullsized":"2023-03-03T06:25:19.461936+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:04.336725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344076+0000","last_change":"2023-03-03T06:25:13.308494+0000","last_active":"2023-03-03T06:25:18.344076+0000","last_peered":"2023-03-03T06:25:18.344076+0000","last_clean":"2023-03-03T06:25:18.344076+0000","last_became_active":"2023-03-03T06:25:13.308110+0000","last_became_peered":"2023-03-03T06:25:13.308110+0000","last_unstale":"2023-03-03T06:25:18.344076+0000","last_undegraded":"2023-03-03T06:25:18.344076+0000","last_fullsized":"2023-03-03T06:25:18.344076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.336445+0000","last_change":"2023-03-03T06:25:13.833412+0000","last_active":"2023-03-03T06:25:18.336445+0000","last_peered":"2023-03-03T06:25:18.336445+0000","last_clean":"2023-03-03T06:25:18.336445+0000","last_became_active":"2023-03-03T06:25:13.833138+0000","last_became_peered":"2023-03-03T06:25:13.833138+0000","last_unstale":"2023-03-03T06:25:18.336445+0000","last_undegraded":"2023-03-03T06:25:18.336445+0000","last_fullsized":"2023-03-03T06:25:18.336445+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:16:17.602774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.876849+0000","last_change":"2023-03-03T06:25:18.354570+0000","last_active":"2023-03-03T06:25:18.876849+0000","last_peered":"2023-03-03T06:25:18.876849+0000","last_clean":"2023-03-03T06:25:18.876849+0000","last_became_active":"2023-03-03T06:25:18.354128+0000","last_became_peered":"2023-03-03T06:25:18.354128+0000","last_unstale":"2023-03-03T06:25:18.876849+0000","last_undegraded":"2023-03-03T06:25:18.876849+0000","last_fullsized":"2023-03-03T06:25:18.876849+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:43:00.515957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335127+0000","last_change":"2023-03-03T06:25:13.307789+0000","last_active":"2023-03-03T06:25:18.335127+0000","last_peered":"2023-03-03T06:25:18.335127+0000","last_clean":"2023-03-03T06:25:18.335127+0000","last_became_active":"2023-03-03T06:25:13.307478+0000","last_became_peered":"2023-03-03T06:25:13.307478+0000","last_unstale":"2023-03-03T06:25:18.335127+0000","last_undegraded":"2023-03-03T06:25:18.335127+0000","last_fullsized":"2023-03-03T06:25:18.335127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:11:32.033289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.339285+0000","last_change":"2023-03-03T06:25:13.830797+0000","last_active":"2023-03-03T06:25:18.339285+0000","last_peered":"2023-03-03T06:25:18.339285+0000","last_clean":"2023-03-03T06:25:18.339285+0000","last_became_active":"2023-03-03T06:25:13.829482+0000","last_became_peered":"2023-03-03T06:25:13.829482+0000","last_unstale":"2023-03-03T06:25:18.339285+0000","last_undegraded":"2023-03-03T06:25:18.339285+0000","last_fullsized":"2023-03-03T06:25:18.339285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:48:01.152785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.459250+0000","last_change":"2023-03-03T06:25:18.358329+0000","last_active":"2023-03-03T06:25:19.459250+0000","last_peered":"2023-03-03T06:25:19.459250+0000","last_clean":"2023-03-03T06:25:19.459250+0000","last_became_active":"2023-03-03T06:25:18.357641+0000","last_became_peered":"2023-03-03T06:25:18.357641+0000","last_unstale":"2023-03-03T06:25:19.459250+0000","last_undegraded":"2023-03-03T06:25:19.459250+0000","last_fullsized":"2023-03-03T06:25:19.459250+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:59.071940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.359353+0000","last_change":"2023-03-03T06:25:13.312665+0000","last_active":"2023-03-03T06:25:18.359353+0000","last_peered":"2023-03-03T06:25:18.359353+0000","last_clean":"2023-03-03T06:25:18.359353+0000","last_became_active":"2023-03-03T06:25:13.312255+0000","last_became_peered":"2023-03-03T06:25:13.312255+0000","last_unstale":"2023-03-03T06:25:18.359353+0000","last_undegraded":"2023-03-03T06:25:18.359353+0000","last_fullsized":"2023-03-03T06:25:18.359353+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344419+0000","last_change":"2023-03-03T06:25:13.847886+0000","last_active":"2023-03-03T06:25:18.344419+0000","last_peered":"2023-03-03T06:25:18.344419+0000","last_clean":"2023-03-03T06:25:18.344419+0000","last_became_active":"2023-03-03T06:25:13.847620+0000","last_became_peered":"2023-03-03T06:25:13.847620+0000","last_unstale":"2023-03-03T06:25:18.344419+0000","last_undegraded":"2023-03-03T06:25:18.344419+0000","last_fullsized":"2023-03-03T06:25:18.344419+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:48:01.152785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.875799+0000","last_change":"2023-03-03T06:25:18.374567+0000","last_active":"2023-03-03T06:25:18.875799+0000","last_peered":"2023-03-03T06:25:18.875799+0000","last_clean":"2023-03-03T06:25:18.875799+0000","last_became_active":"2023-03-03T06:25:18.374172+0000","last_became_peered":"2023-03-03T06:25:18.374172+0000","last_unstale":"2023-03-03T06:25:18.875799+0000","last_undegraded":"2023-03-03T06:25:18.875799+0000","last_fullsized":"2023-03-03T06:25:18.875799+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:11:32.109566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341581+0000","last_change":"2023-03-03T06:25:13.310405+0000","last_active":"2023-03-03T06:25:18.341581+0000","last_peered":"2023-03-03T06:25:18.341581+0000","last_clean":"2023-03-03T06:25:18.341581+0000","last_became_active":"2023-03-03T06:25:13.310029+0000","last_became_peered":"2023-03-03T06:25:13.310029+0000","last_unstale":"2023-03-03T06:25:18.341581+0000","last_undegraded":"2023-03-03T06:25:18.341581+0000","last_fullsized":"2023-03-03T06:25:18.341581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:16:16.586666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343606+0000","last_change":"2023-03-03T06:25:13.839392+0000","last_active":"2023-03-03T06:25:18.343606+0000","last_peered":"2023-03-03T06:25:18.343606+0000","last_clean":"2023-03-03T06:25:18.343606+0000","last_became_active":"2023-03-03T06:25:13.839078+0000","last_became_peered":"2023-03-03T06:25:13.839078+0000","last_unstale":"2023-03-03T06:25:18.343606+0000","last_undegraded":"2023-03-03T06:25:18.343606+0000","last_fullsized":"2023-03-03T06:25:18.343606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:42:30.992559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341473+0000","last_change":"2023-03-03T06:25:13.836064+0000","last_active":"2023-03-03T06:25:18.341473+0000","last_peered":"2023-03-03T06:25:18.341473+0000","last_clean":"2023-03-03T06:25:18.341473+0000","last_became_active":"2023-03-03T06:25:13.835743+0000","last_became_peered":"2023-03-03T06:25:13.835743+0000","last_unstale":"2023-03-03T06:25:18.341473+0000","last_undegraded":"2023-03-03T06:25:18.341473+0000","last_fullsized":"2023-03-03T06:25:18.341473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:52:13.551129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343523+0000","last_change":"2023-03-03T06:25:13.293588+0000","last_active":"2023-03-03T06:25:18.343523+0000","last_peered":"2023-03-03T06:25:18.343523+0000","last_clean":"2023-03-03T06:25:18.343523+0000","last_became_active":"2023-03-03T06:25:13.292998+0000","last_became_peered":"2023-03-03T06:25:13.292998+0000","last_unstale":"2023-03-03T06:25:18.343523+0000","last_undegraded":"2023-03-03T06:25:18.343523+0000","last_fullsized":"2023-03-03T06:25:18.343523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:17:34.388274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.459022+0000","last_change":"2023-03-03T06:25:18.365923+0000","last_active":"2023-03-03T06:25:19.459022+0000","last_peered":"2023-03-03T06:25:19.459022+0000","last_clean":"2023-03-03T06:25:19.459022+0000","last_became_active":"2023-03-03T06:25:18.365397+0000","last_became_peered":"2023-03-03T06:25:18.365397+0000","last_unstale":"2023-03-03T06:25:19.459022+0000","last_undegraded":"2023-03-03T06:25:19.459022+0000","last_fullsized":"2023-03-03T06:25:19.459022+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:45:30.096214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337824+0000","last_change":"2023-03-03T06:25:13.837989+0000","last_active":"2023-03-03T06:25:18.337824+0000","last_peered":"2023-03-03T06:25:18.337824+0000","last_clean":"2023-03-03T06:25:18.337824+0000","last_became_active":"2023-03-03T06:25:13.837724+0000","last_became_peered":"2023-03-03T06:25:13.837724+0000","last_unstale":"2023-03-03T06:25:18.337824+0000","last_undegraded":"2023-03-03T06:25:18.337824+0000","last_fullsized":"2023-03-03T06:25:18.337824+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353186+0000","last_change":"2023-03-03T06:25:13.302153+0000","last_active":"2023-03-03T06:25:18.353186+0000","last_peered":"2023-03-03T06:25:18.353186+0000","last_clean":"2023-03-03T06:25:18.353186+0000","last_became_active":"2023-03-03T06:25:13.301751+0000","last_became_peered":"2023-03-03T06:25:13.301751+0000","last_unstale":"2023-03-03T06:25:18.353186+0000","last_undegraded":"2023-03-03T06:25:18.353186+0000","last_fullsized":"2023-03-03T06:25:18.353186+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.462897+0000","last_change":"2023-03-03T06:25:18.366010+0000","last_active":"2023-03-03T06:25:19.462897+0000","last_peered":"2023-03-03T06:25:19.462897+0000","last_clean":"2023-03-03T06:25:19.462897+0000","last_became_active":"2023-03-03T06:25:18.362457+0000","last_became_peered":"2023-03-03T06:25:18.362457+0000","last_unstale":"2023-03-03T06:25:19.462897+0000","last_undegraded":"2023-03-03T06:25:19.462897+0000","last_fullsized":"2023-03-03T06:25:19.462897+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:17:02.442524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350698+0000","last_change":"2023-03-03T06:25:13.312520+0000","last_active":"2023-03-03T06:25:18.350698+0000","last_peered":"2023-03-03T06:25:18.350698+0000","last_clean":"2023-03-03T06:25:18.350698+0000","last_became_active":"2023-03-03T06:25:13.312105+0000","last_became_peered":"2023-03-03T06:25:13.312105+0000","last_unstale":"2023-03-03T06:25:18.350698+0000","last_undegraded":"2023-03-03T06:25:18.350698+0000","last_fullsized":"2023-03-03T06:25:18.350698+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:51:29.967970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358702+0000","last_change":"2023-03-03T06:25:13.835492+0000","last_active":"2023-03-03T06:25:18.358702+0000","last_peered":"2023-03-03T06:25:18.358702+0000","last_clean":"2023-03-03T06:25:18.358702+0000","last_became_active":"2023-03-03T06:25:13.835173+0000","last_became_peered":"2023-03-03T06:25:13.835173+0000","last_unstale":"2023-03-03T06:25:18.358702+0000","last_undegraded":"2023-03-03T06:25:18.358702+0000","last_fullsized":"2023-03-03T06:25:18.358702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467726+0000","last_change":"2023-03-03T06:25:18.362177+0000","last_active":"2023-03-03T06:25:19.467726+0000","last_peered":"2023-03-03T06:25:19.467726+0000","last_clean":"2023-03-03T06:25:19.467726+0000","last_became_active":"2023-03-03T06:25:18.361833+0000","last_became_peered":"2023-03-03T06:25:18.361833+0000","last_unstale":"2023-03-03T06:25:19.467726+0000","last_undegraded":"2023-03-03T06:25:19.467726+0000","last_fullsized":"2023-03-03T06:25:19.467726+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:04:44.947509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:32.598145+0000","last_change":"2023-03-03T06:25:13.302558+0000","last_active":"2023-03-03T06:25:32.598145+0000","last_peered":"2023-03-03T06:25:32.598145+0000","last_clean":"2023-03-03T06:25:32.598145+0000","last_became_active":"2023-03-03T06:25:13.301983+0000","last_became_peered":"2023-03-03T06:25:13.301983+0000","last_unstale":"2023-03-03T06:25:32.598145+0000","last_undegraded":"2023-03-03T06:25:32.598145+0000","last_fullsized":"2023-03-03T06:25:32.598145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:34:41.158814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3745,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334685+0000","last_change":"2023-03-03T06:25:13.835359+0000","last_active":"2023-03-03T06:25:18.334685+0000","last_peered":"2023-03-03T06:25:18.334685+0000","last_clean":"2023-03-03T06:25:18.334685+0000","last_became_active":"2023-03-03T06:25:13.835103+0000","last_became_peered":"2023-03-03T06:25:13.835103+0000","last_unstale":"2023-03-03T06:25:18.334685+0000","last_undegraded":"2023-03-03T06:25:18.334685+0000","last_fullsized":"2023-03-03T06:25:18.334685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461833+0000","last_change":"2023-03-03T06:25:18.376174+0000","last_active":"2023-03-03T06:25:19.461833+0000","last_peered":"2023-03-03T06:25:19.461833+0000","last_clean":"2023-03-03T06:25:19.461833+0000","last_became_active":"2023-03-03T06:25:18.375779+0000","last_became_peered":"2023-03-03T06:25:18.375779+0000","last_unstale":"2023-03-03T06:25:19.461833+0000","last_undegraded":"2023-03-03T06:25:19.461833+0000","last_fullsized":"2023-03-03T06:25:19.461833+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:40:49.542836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335280+0000","last_change":"2023-03-03T06:25:13.837277+0000","last_active":"2023-03-03T06:25:18.335280+0000","last_peered":"2023-03-03T06:25:18.335280+0000","last_clean":"2023-03-03T06:25:18.335280+0000","last_became_active":"2023-03-03T06:25:13.836510+0000","last_became_peered":"2023-03-03T06:25:13.836510+0000","last_unstale":"2023-03-03T06:25:18.335280+0000","last_undegraded":"2023-03-03T06:25:18.335280+0000","last_fullsized":"2023-03-03T06:25:18.335280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343682+0000","last_change":"2023-03-03T06:25:07.205700+0000","last_active":"2023-03-03T06:25:18.343682+0000","last_peered":"2023-03-03T06:25:18.343682+0000","last_clean":"2023-03-03T06:25:18.343682+0000","last_became_active":"2023-03-03T06:25:07.205269+0000","last_became_peered":"2023-03-03T06:25:07.205269+0000","last_unstale":"2023-03-03T06:25:18.343682+0000","last_undegraded":"2023-03-03T06:25:18.343682+0000","last_fullsized":"2023-03-03T06:25:18.343682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:17:53.442570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344712+0000","last_change":"2023-03-03T06:25:13.309537+0000","last_active":"2023-03-03T06:25:18.344712+0000","last_peered":"2023-03-03T06:25:18.344712+0000","last_clean":"2023-03-03T06:25:18.344712+0000","last_became_active":"2023-03-03T06:25:13.309208+0000","last_became_peered":"2023-03-03T06:25:13.309208+0000","last_unstale":"2023-03-03T06:25:18.344712+0000","last_undegraded":"2023-03-03T06:25:18.344712+0000","last_fullsized":"2023-03-03T06:25:18.344712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:11:45.788884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467622+0000","last_change":"2023-03-03T06:25:18.367070+0000","last_active":"2023-03-03T06:25:19.467622+0000","last_peered":"2023-03-03T06:25:19.467622+0000","last_clean":"2023-03-03T06:25:19.467622+0000","last_became_active":"2023-03-03T06:25:18.366674+0000","last_became_peered":"2023-03-03T06:25:18.366674+0000","last_unstale":"2023-03-03T06:25:19.467622+0000","last_undegraded":"2023-03-03T06:25:19.467622+0000","last_fullsized":"2023-03-03T06:25:19.467622+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:43:20.174095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341556+0000","last_change":"2023-03-03T06:25:13.837256+0000","last_active":"2023-03-03T06:25:18.341556+0000","last_peered":"2023-03-03T06:25:18.341556+0000","last_clean":"2023-03-03T06:25:18.341556+0000","last_became_active":"2023-03-03T06:25:13.836942+0000","last_became_peered":"2023-03-03T06:25:13.836942+0000","last_unstale":"2023-03-03T06:25:18.341556+0000","last_undegraded":"2023-03-03T06:25:18.341556+0000","last_fullsized":"2023-03-03T06:25:18.341556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:05:53.754239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357705+0000","last_change":"2023-03-03T06:25:07.204903+0000","last_active":"2023-03-03T06:25:18.357705+0000","last_peered":"2023-03-03T06:25:18.357705+0000","last_clean":"2023-03-03T06:25:18.357705+0000","last_became_active":"2023-03-03T06:25:07.204601+0000","last_became_peered":"2023-03-03T06:25:07.204601+0000","last_unstale":"2023-03-03T06:25:18.357705+0000","last_undegraded":"2023-03-03T06:25:18.357705+0000","last_fullsized":"2023-03-03T06:25:18.357705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:18:36.947912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343364+0000","last_change":"2023-03-03T06:25:13.304543+0000","last_active":"2023-03-03T06:25:18.343364+0000","last_peered":"2023-03-03T06:25:18.343364+0000","last_clean":"2023-03-03T06:25:18.343364+0000","last_became_active":"2023-03-03T06:25:13.304142+0000","last_became_peered":"2023-03-03T06:25:13.304142+0000","last_unstale":"2023-03-03T06:25:18.343364+0000","last_undegraded":"2023-03-03T06:25:18.343364+0000","last_fullsized":"2023-03-03T06:25:18.343364+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.876556+0000","last_change":"2023-03-03T06:25:18.354139+0000","last_active":"2023-03-03T06:25:18.876556+0000","last_peered":"2023-03-03T06:25:18.876556+0000","last_clean":"2023-03-03T06:25:18.876556+0000","last_became_active":"2023-03-03T06:25:18.353543+0000","last_became_peered":"2023-03-03T06:25:18.353543+0000","last_unstale":"2023-03-03T06:25:18.876556+0000","last_undegraded":"2023-03-03T06:25:18.876556+0000","last_fullsized":"2023-03-03T06:25:18.876556+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:11:23.940316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343906+0000","last_change":"2023-03-03T06:25:13.838819+0000","last_active":"2023-03-03T06:25:18.343906+0000","last_peered":"2023-03-03T06:25:18.343906+0000","last_clean":"2023-03-03T06:25:18.343906+0000","last_became_active":"2023-03-03T06:25:13.838558+0000","last_became_peered":"2023-03-03T06:25:13.838558+0000","last_unstale":"2023-03-03T06:25:18.343906+0000","last_undegraded":"2023-03-03T06:25:18.343906+0000","last_fullsized":"2023-03-03T06:25:18.343906+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350807+0000","last_change":"2023-03-03T06:25:07.208473+0000","last_active":"2023-03-03T06:25:18.350807+0000","last_peered":"2023-03-03T06:25:18.350807+0000","last_clean":"2023-03-03T06:25:18.350807+0000","last_became_active":"2023-03-03T06:25:07.208041+0000","last_became_peered":"2023-03-03T06:25:07.208041+0000","last_unstale":"2023-03-03T06:25:18.350807+0000","last_undegraded":"2023-03-03T06:25:18.350807+0000","last_fullsized":"2023-03-03T06:25:18.350807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:22:19.852493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357626+0000","last_change":"2023-03-03T06:25:13.308259+0000","last_active":"2023-03-03T06:25:18.357626+0000","last_peered":"2023-03-03T06:25:18.357626+0000","last_clean":"2023-03-03T06:25:18.357626+0000","last_became_active":"2023-03-03T06:25:13.307902+0000","last_became_peered":"2023-03-03T06:25:13.307902+0000","last_unstale":"2023-03-03T06:25:18.357626+0000","last_undegraded":"2023-03-03T06:25:18.357626+0000","last_fullsized":"2023-03-03T06:25:18.357626+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:08.271702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.459074+0000","last_change":"2023-03-03T06:25:18.355878+0000","last_active":"2023-03-03T06:25:19.459074+0000","last_peered":"2023-03-03T06:25:19.459074+0000","last_clean":"2023-03-03T06:25:19.459074+0000","last_became_active":"2023-03-03T06:25:18.355460+0000","last_became_peered":"2023-03-03T06:25:18.355460+0000","last_unstale":"2023-03-03T06:25:19.459074+0000","last_undegraded":"2023-03-03T06:25:19.459074+0000","last_fullsized":"2023-03-03T06:25:19.459074+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:38:51.150150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358771+0000","last_change":"2023-03-03T06:25:13.839947+0000","last_active":"2023-03-03T06:25:18.358771+0000","last_peered":"2023-03-03T06:25:18.358771+0000","last_clean":"2023-03-03T06:25:18.358771+0000","last_became_active":"2023-03-03T06:25:13.839632+0000","last_became_peered":"2023-03-03T06:25:13.839632+0000","last_unstale":"2023-03-03T06:25:18.358771+0000","last_undegraded":"2023-03-03T06:25:18.358771+0000","last_fullsized":"2023-03-03T06:25:18.358771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357780+0000","last_change":"2023-03-03T06:25:07.201865+0000","last_active":"2023-03-03T06:25:18.357780+0000","last_peered":"2023-03-03T06:25:18.357780+0000","last_clean":"2023-03-03T06:25:18.357780+0000","last_became_active":"2023-03-03T06:25:07.201411+0000","last_became_peered":"2023-03-03T06:25:07.201411+0000","last_unstale":"2023-03-03T06:25:18.357780+0000","last_undegraded":"2023-03-03T06:25:18.357780+0000","last_fullsized":"2023-03-03T06:25:18.357780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:17:53.442570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352970+0000","last_change":"2023-03-03T06:25:13.312426+0000","last_active":"2023-03-03T06:25:18.352970+0000","last_peered":"2023-03-03T06:25:18.352970+0000","last_clean":"2023-03-03T06:25:18.352970+0000","last_became_active":"2023-03-03T06:25:13.312129+0000","last_became_peered":"2023-03-03T06:25:13.312129+0000","last_unstale":"2023-03-03T06:25:18.352970+0000","last_undegraded":"2023-03-03T06:25:18.352970+0000","last_fullsized":"2023-03-03T06:25:18.352970+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.875532+0000","last_change":"2023-03-03T06:25:18.365085+0000","last_active":"2023-03-03T06:25:18.875532+0000","last_peered":"2023-03-03T06:25:18.875532+0000","last_clean":"2023-03-03T06:25:18.875532+0000","last_became_active":"2023-03-03T06:25:18.364690+0000","last_became_peered":"2023-03-03T06:25:18.364690+0000","last_unstale":"2023-03-03T06:25:18.875532+0000","last_undegraded":"2023-03-03T06:25:18.875532+0000","last_fullsized":"2023-03-03T06:25:18.875532+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:30.336080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357548+0000","last_change":"2023-03-03T06:25:13.828855+0000","last_active":"2023-03-03T06:25:18.357548+0000","last_peered":"2023-03-03T06:25:18.357548+0000","last_clean":"2023-03-03T06:25:18.357548+0000","last_became_active":"2023-03-03T06:25:13.828546+0000","last_became_peered":"2023-03-03T06:25:13.828546+0000","last_unstale":"2023-03-03T06:25:18.357548+0000","last_undegraded":"2023-03-03T06:25:18.357548+0000","last_fullsized":"2023-03-03T06:25:18.357548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334824+0000","last_change":"2023-03-03T06:25:13.298859+0000","last_active":"2023-03-03T06:25:18.334824+0000","last_peered":"2023-03-03T06:25:18.334824+0000","last_clean":"2023-03-03T06:25:18.334824+0000","last_became_active":"2023-03-03T06:25:13.298586+0000","last_became_peered":"2023-03-03T06:25:13.298586+0000","last_unstale":"2023-03-03T06:25:18.334824+0000","last_undegraded":"2023-03-03T06:25:18.334824+0000","last_fullsized":"2023-03-03T06:25:18.334824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:11:45.788884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353089+0000","last_change":"2023-03-03T06:25:07.201941+0000","last_active":"2023-03-03T06:25:18.353089+0000","last_peered":"2023-03-03T06:25:18.353089+0000","last_clean":"2023-03-03T06:25:18.353089+0000","last_became_active":"2023-03-03T06:25:07.201437+0000","last_became_peered":"2023-03-03T06:25:07.201437+0000","last_unstale":"2023-03-03T06:25:18.353089+0000","last_undegraded":"2023-03-03T06:25:18.353089+0000","last_fullsized":"2023-03-03T06:25:18.353089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:48:52.145810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.875719+0000","last_change":"2023-03-03T06:25:18.374636+0000","last_active":"2023-03-03T06:25:18.875719+0000","last_peered":"2023-03-03T06:25:18.875719+0000","last_clean":"2023-03-03T06:25:18.875719+0000","last_became_active":"2023-03-03T06:25:18.373925+0000","last_became_peered":"2023-03-03T06:25:18.373925+0000","last_unstale":"2023-03-03T06:25:18.875719+0000","last_undegraded":"2023-03-03T06:25:18.875719+0000","last_fullsized":"2023-03-03T06:25:18.875719+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:07:38.561763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.359409+0000","last_change":"2023-03-03T06:25:13.833104+0000","last_active":"2023-03-03T06:25:18.359409+0000","last_peered":"2023-03-03T06:25:18.359409+0000","last_clean":"2023-03-03T06:25:18.359409+0000","last_became_active":"2023-03-03T06:25:13.832815+0000","last_became_peered":"2023-03-03T06:25:13.832815+0000","last_unstale":"2023-03-03T06:25:18.359409+0000","last_undegraded":"2023-03-03T06:25:18.359409+0000","last_fullsized":"2023-03-03T06:25:18.359409+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.356972+0000","last_change":"2023-03-03T06:25:13.298339+0000","last_active":"2023-03-03T06:25:18.356972+0000","last_peered":"2023-03-03T06:25:18.356972+0000","last_clean":"2023-03-03T06:25:18.356972+0000","last_became_active":"2023-03-03T06:25:13.297116+0000","last_became_peered":"2023-03-03T06:25:13.297116+0000","last_unstale":"2023-03-03T06:25:18.356972+0000","last_undegraded":"2023-03-03T06:25:18.356972+0000","last_fullsized":"2023-03-03T06:25:18.356972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335947+0000","last_change":"2023-03-03T06:25:07.437275+0000","last_active":"2023-03-03T06:25:18.335947+0000","last_peered":"2023-03-03T06:25:18.335947+0000","last_clean":"2023-03-03T06:25:18.335947+0000","last_became_active":"2023-03-03T06:25:07.436823+0000","last_became_peered":"2023-03-03T06:25:07.436823+0000","last_unstale":"2023-03-03T06:25:18.335947+0000","last_undegraded":"2023-03-03T06:25:18.335947+0000","last_fullsized":"2023-03-03T06:25:18.335947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:24:15.529041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217928+0000","last_change":"2023-03-03T06:25:18.361949+0000","last_active":"2023-03-03T06:25:19.217928+0000","last_peered":"2023-03-03T06:25:19.217928+0000","last_clean":"2023-03-03T06:25:19.217928+0000","last_became_active":"2023-03-03T06:25:18.361407+0000","last_became_peered":"2023-03-03T06:25:18.361407+0000","last_unstale":"2023-03-03T06:25:19.217928+0000","last_undegraded":"2023-03-03T06:25:19.217928+0000","last_fullsized":"2023-03-03T06:25:19.217928+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:59.071940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344511+0000","last_change":"2023-03-03T06:25:13.833788+0000","last_active":"2023-03-03T06:25:18.344511+0000","last_peered":"2023-03-03T06:25:18.344511+0000","last_clean":"2023-03-03T06:25:18.344511+0000","last_became_active":"2023-03-03T06:25:13.833532+0000","last_became_peered":"2023-03-03T06:25:13.833532+0000","last_unstale":"2023-03-03T06:25:18.344511+0000","last_undegraded":"2023-03-03T06:25:18.344511+0000","last_fullsized":"2023-03-03T06:25:18.344511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:59:50.232210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342831+0000","last_change":"2023-03-03T06:25:13.308357+0000","last_active":"2023-03-03T06:25:18.342831+0000","last_peered":"2023-03-03T06:25:18.342831+0000","last_clean":"2023-03-03T06:25:18.342831+0000","last_became_active":"2023-03-03T06:25:13.307981+0000","last_became_peered":"2023-03-03T06:25:13.307981+0000","last_unstale":"2023-03-03T06:25:18.342831+0000","last_undegraded":"2023-03-03T06:25:18.342831+0000","last_fullsized":"2023-03-03T06:25:18.342831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346873+0000","last_change":"2023-03-03T06:25:07.208585+0000","last_active":"2023-03-03T06:25:18.346873+0000","last_peered":"2023-03-03T06:25:18.346873+0000","last_clean":"2023-03-03T06:25:18.346873+0000","last_became_active":"2023-03-03T06:25:07.208257+0000","last_became_peered":"2023-03-03T06:25:07.208257+0000","last_unstale":"2023-03-03T06:25:18.346873+0000","last_undegraded":"2023-03-03T06:25:18.346873+0000","last_fullsized":"2023-03-03T06:25:18.346873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:11:52.893733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879358+0000","last_change":"2023-03-03T06:25:18.369960+0000","last_active":"2023-03-03T06:25:18.879358+0000","last_peered":"2023-03-03T06:25:18.879358+0000","last_clean":"2023-03-03T06:25:18.879358+0000","last_became_active":"2023-03-03T06:25:18.367846+0000","last_became_peered":"2023-03-03T06:25:18.367846+0000","last_unstale":"2023-03-03T06:25:18.879358+0000","last_undegraded":"2023-03-03T06:25:18.879358+0000","last_fullsized":"2023-03-03T06:25:18.879358+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:12:09.571057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342233+0000","last_change":"2023-03-03T06:25:13.857729+0000","last_active":"2023-03-03T06:25:18.342233+0000","last_peered":"2023-03-03T06:25:18.342233+0000","last_clean":"2023-03-03T06:25:18.342233+0000","last_became_active":"2023-03-03T06:25:13.857453+0000","last_became_peered":"2023-03-03T06:25:13.857453+0000","last_unstale":"2023-03-03T06:25:18.342233+0000","last_undegraded":"2023-03-03T06:25:18.342233+0000","last_fullsized":"2023-03-03T06:25:18.342233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:46:08.313482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.339103+0000","last_change":"2023-03-03T06:25:13.303419+0000","last_active":"2023-03-03T06:25:18.339103+0000","last_peered":"2023-03-03T06:25:18.339103+0000","last_clean":"2023-03-03T06:25:18.339103+0000","last_became_active":"2023-03-03T06:25:13.303084+0000","last_became_peered":"2023-03-03T06:25:13.303084+0000","last_unstale":"2023-03-03T06:25:18.339103+0000","last_undegraded":"2023-03-03T06:25:18.339103+0000","last_fullsized":"2023-03-03T06:25:18.339103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:07.106860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352430+0000","last_change":"2023-03-03T06:25:07.203115+0000","last_active":"2023-03-03T06:25:18.352430+0000","last_peered":"2023-03-03T06:25:18.352430+0000","last_clean":"2023-03-03T06:25:18.352430+0000","last_became_active":"2023-03-03T06:25:07.202705+0000","last_became_peered":"2023-03-03T06:25:07.202705+0000","last_unstale":"2023-03-03T06:25:18.352430+0000","last_undegraded":"2023-03-03T06:25:18.352430+0000","last_fullsized":"2023-03-03T06:25:18.352430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:11:32.938698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346951+0000","last_change":"2023-03-03T06:25:02.138688+0000","last_active":"2023-03-03T06:25:18.346951+0000","last_peered":"2023-03-03T06:25:18.346951+0000","last_clean":"2023-03-03T06:25:18.346951+0000","last_became_active":"2023-03-03T06:25:02.137945+0000","last_became_peered":"2023-03-03T06:25:02.137945+0000","last_unstale":"2023-03-03T06:25:18.346951+0000","last_undegraded":"2023-03-03T06:25:18.346951+0000","last_fullsized":"2023-03-03T06:25:18.346951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:00.679233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:00.679233+0000","last_clean_scrub_stamp":"2023-03-03T06:25:00.679233+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:24:46.304379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.463093+0000","last_change":"2023-03-03T06:25:18.368371+0000","last_active":"2023-03-03T06:25:19.463093+0000","last_peered":"2023-03-03T06:25:19.463093+0000","last_clean":"2023-03-03T06:25:19.463093+0000","last_became_active":"2023-03-03T06:25:18.367896+0000","last_became_peered":"2023-03-03T06:25:18.367896+0000","last_unstale":"2023-03-03T06:25:19.463093+0000","last_undegraded":"2023-03-03T06:25:19.463093+0000","last_fullsized":"2023-03-03T06:25:19.463093+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:43:20.174095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358378+0000","last_change":"2023-03-03T06:25:13.842311+0000","last_active":"2023-03-03T06:25:18.358378+0000","last_peered":"2023-03-03T06:25:18.358378+0000","last_clean":"2023-03-03T06:25:18.358378+0000","last_became_active":"2023-03-03T06:25:13.841952+0000","last_became_peered":"2023-03-03T06:25:13.841952+0000","last_unstale":"2023-03-03T06:25:18.358378+0000","last_undegraded":"2023-03-03T06:25:18.358378+0000","last_fullsized":"2023-03-03T06:25:18.358378+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:46:08.313482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341679+0000","last_change":"2023-03-03T06:25:13.306584+0000","last_active":"2023-03-03T06:25:18.341679+0000","last_peered":"2023-03-03T06:25:18.341679+0000","last_clean":"2023-03-03T06:25:18.341679+0000","last_became_active":"2023-03-03T06:25:13.306295+0000","last_became_peered":"2023-03-03T06:25:13.306295+0000","last_unstale":"2023-03-03T06:25:18.341679+0000","last_undegraded":"2023-03-03T06:25:18.341679+0000","last_fullsized":"2023-03-03T06:25:18.341679+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.878155+0000","last_change":"2023-03-03T06:25:18.361381+0000","last_active":"2023-03-03T06:25:18.878155+0000","last_peered":"2023-03-03T06:25:18.878155+0000","last_clean":"2023-03-03T06:25:18.878155+0000","last_became_active":"2023-03-03T06:25:18.360819+0000","last_became_peered":"2023-03-03T06:25:18.360819+0000","last_unstale":"2023-03-03T06:25:18.878155+0000","last_undegraded":"2023-03-03T06:25:18.878155+0000","last_fullsized":"2023-03-03T06:25:18.878155+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:39:21.927192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335650+0000","last_change":"2023-03-03T06:25:13.848629+0000","last_active":"2023-03-03T06:25:18.335650+0000","last_peered":"2023-03-03T06:25:18.335650+0000","last_clean":"2023-03-03T06:25:18.335650+0000","last_became_active":"2023-03-03T06:25:13.848353+0000","last_became_peered":"2023-03-03T06:25:13.848353+0000","last_unstale":"2023-03-03T06:25:18.335650+0000","last_undegraded":"2023-03-03T06:25:18.335650+0000","last_fullsized":"2023-03-03T06:25:18.335650+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:59:50.232210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335310+0000","last_change":"2023-03-03T06:25:13.297292+0000","last_active":"2023-03-03T06:25:18.335310+0000","last_peered":"2023-03-03T06:25:18.335310+0000","last_clean":"2023-03-03T06:25:18.335310+0000","last_became_active":"2023-03-03T06:25:13.296692+0000","last_became_peered":"2023-03-03T06:25:13.296692+0000","last_unstale":"2023-03-03T06:25:18.335310+0000","last_undegraded":"2023-03-03T06:25:18.335310+0000","last_fullsized":"2023-03-03T06:25:18.335310+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:09:43.297428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879361+0000","last_change":"2023-03-03T06:25:18.369017+0000","last_active":"2023-03-03T06:25:18.879361+0000","last_peered":"2023-03-03T06:25:18.879361+0000","last_clean":"2023-03-03T06:25:18.879361+0000","last_became_active":"2023-03-03T06:25:18.366873+0000","last_became_peered":"2023-03-03T06:25:18.366873+0000","last_unstale":"2023-03-03T06:25:18.879361+0000","last_undegraded":"2023-03-03T06:25:18.879361+0000","last_fullsized":"2023-03-03T06:25:18.879361+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:17:02.442524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334965+0000","last_change":"2023-03-03T06:25:13.855863+0000","last_active":"2023-03-03T06:25:18.334965+0000","last_peered":"2023-03-03T06:25:18.334965+0000","last_clean":"2023-03-03T06:25:18.334965+0000","last_became_active":"2023-03-03T06:25:13.855565+0000","last_became_peered":"2023-03-03T06:25:13.855565+0000","last_unstale":"2023-03-03T06:25:18.334965+0000","last_undegraded":"2023-03-03T06:25:18.334965+0000","last_fullsized":"2023-03-03T06:25:18.334965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:45:26.049658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341408+0000","last_change":"2023-03-03T06:25:13.306690+0000","last_active":"2023-03-03T06:25:18.341408+0000","last_peered":"2023-03-03T06:25:18.341408+0000","last_clean":"2023-03-03T06:25:18.341408+0000","last_became_active":"2023-03-03T06:25:13.306348+0000","last_became_peered":"2023-03-03T06:25:13.306348+0000","last_unstale":"2023-03-03T06:25:18.341408+0000","last_undegraded":"2023-03-03T06:25:18.341408+0000","last_fullsized":"2023-03-03T06:25:18.341408+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:05.884433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.461865+0000","last_change":"2023-03-03T06:25:18.373926+0000","last_active":"2023-03-03T06:25:19.461865+0000","last_peered":"2023-03-03T06:25:19.461865+0000","last_clean":"2023-03-03T06:25:19.461865+0000","last_became_active":"2023-03-03T06:25:18.373371+0000","last_became_peered":"2023-03-03T06:25:18.373371+0000","last_unstale":"2023-03-03T06:25:19.461865+0000","last_undegraded":"2023-03-03T06:25:19.461865+0000","last_fullsized":"2023-03-03T06:25:19.461865+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:17:02.442524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335386+0000","last_change":"2023-03-03T06:25:13.846359+0000","last_active":"2023-03-03T06:25:18.335386+0000","last_peered":"2023-03-03T06:25:18.335386+0000","last_clean":"2023-03-03T06:25:18.335386+0000","last_became_active":"2023-03-03T06:25:13.845966+0000","last_became_peered":"2023-03-03T06:25:13.845966+0000","last_unstale":"2023-03-03T06:25:18.335386+0000","last_undegraded":"2023-03-03T06:25:18.335386+0000","last_fullsized":"2023-03-03T06:25:18.335386+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:38:43.138896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341996+0000","last_change":"2023-03-03T06:25:13.310274+0000","last_active":"2023-03-03T06:25:18.341996+0000","last_peered":"2023-03-03T06:25:18.341996+0000","last_clean":"2023-03-03T06:25:18.341996+0000","last_became_active":"2023-03-03T06:25:13.310005+0000","last_became_peered":"2023-03-03T06:25:13.310005+0000","last_unstale":"2023-03-03T06:25:18.341996+0000","last_undegraded":"2023-03-03T06:25:18.341996+0000","last_fullsized":"2023-03-03T06:25:18.341996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217857+0000","last_change":"2023-03-03T06:25:18.362444+0000","last_active":"2023-03-03T06:25:19.217857+0000","last_peered":"2023-03-03T06:25:19.217857+0000","last_clean":"2023-03-03T06:25:19.217857+0000","last_became_active":"2023-03-03T06:25:18.362087+0000","last_became_peered":"2023-03-03T06:25:18.362087+0000","last_unstale":"2023-03-03T06:25:19.217857+0000","last_undegraded":"2023-03-03T06:25:19.217857+0000","last_fullsized":"2023-03-03T06:25:19.217857+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:52:17.597685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343862+0000","last_change":"2023-03-03T06:25:13.834113+0000","last_active":"2023-03-03T06:25:18.343862+0000","last_peered":"2023-03-03T06:25:18.343862+0000","last_clean":"2023-03-03T06:25:18.343862+0000","last_became_active":"2023-03-03T06:25:13.833874+0000","last_became_peered":"2023-03-03T06:25:13.833874+0000","last_unstale":"2023-03-03T06:25:18.343862+0000","last_undegraded":"2023-03-03T06:25:18.343862+0000","last_fullsized":"2023-03-03T06:25:18.343862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352647+0000","last_change":"2023-03-03T06:25:13.310942+0000","last_active":"2023-03-03T06:25:18.352647+0000","last_peered":"2023-03-03T06:25:18.352647+0000","last_clean":"2023-03-03T06:25:18.352647+0000","last_became_active":"2023-03-03T06:25:13.310632+0000","last_became_peered":"2023-03-03T06:25:13.310632+0000","last_unstale":"2023-03-03T06:25:18.352647+0000","last_undegraded":"2023-03-03T06:25:18.352647+0000","last_fullsized":"2023-03-03T06:25:18.352647+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:16:16.586666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217181+0000","last_change":"2023-03-03T06:25:18.360063+0000","last_active":"2023-03-03T06:25:19.217181+0000","last_peered":"2023-03-03T06:25:19.217181+0000","last_clean":"2023-03-03T06:25:19.217181+0000","last_became_active":"2023-03-03T06:25:18.359549+0000","last_became_peered":"2023-03-03T06:25:18.359549+0000","last_unstale":"2023-03-03T06:25:19.217181+0000","last_undegraded":"2023-03-03T06:25:19.217181+0000","last_fullsized":"2023-03-03T06:25:19.217181+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:30:28.010892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341203+0000","last_change":"2023-03-03T06:25:13.844840+0000","last_active":"2023-03-03T06:25:18.341203+0000","last_peered":"2023-03-03T06:25:18.341203+0000","last_clean":"2023-03-03T06:25:18.341203+0000","last_became_active":"2023-03-03T06:25:13.844556+0000","last_became_peered":"2023-03-03T06:25:13.844556+0000","last_unstale":"2023-03-03T06:25:18.341203+0000","last_undegraded":"2023-03-03T06:25:18.341203+0000","last_fullsized":"2023-03-03T06:25:18.341203+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:19:49.547542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350702+0000","last_change":"2023-03-03T06:25:13.303131+0000","last_active":"2023-03-03T06:25:18.350702+0000","last_peered":"2023-03-03T06:25:18.350702+0000","last_clean":"2023-03-03T06:25:18.350702+0000","last_became_active":"2023-03-03T06:25:13.302861+0000","last_became_peered":"2023-03-03T06:25:13.302861+0000","last_unstale":"2023-03-03T06:25:18.350702+0000","last_undegraded":"2023-03-03T06:25:18.350702+0000","last_fullsized":"2023-03-03T06:25:18.350702+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:46:07.297374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217444+0000","last_change":"2023-03-03T06:25:18.363570+0000","last_active":"2023-03-03T06:25:19.217444+0000","last_peered":"2023-03-03T06:25:19.217444+0000","last_clean":"2023-03-03T06:25:19.217444+0000","last_became_active":"2023-03-03T06:25:18.363224+0000","last_became_peered":"2023-03-03T06:25:18.363224+0000","last_unstale":"2023-03-03T06:25:19.217444+0000","last_undegraded":"2023-03-03T06:25:19.217444+0000","last_fullsized":"2023-03-03T06:25:19.217444+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:38:51.150150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335118+0000","last_change":"2023-03-03T06:25:13.837508+0000","last_active":"2023-03-03T06:25:18.335118+0000","last_peered":"2023-03-03T06:25:18.335118+0000","last_clean":"2023-03-03T06:25:18.335118+0000","last_became_active":"2023-03-03T06:25:13.836864+0000","last_became_peered":"2023-03-03T06:25:13.836864+0000","last_unstale":"2023-03-03T06:25:18.335118+0000","last_undegraded":"2023-03-03T06:25:18.335118+0000","last_fullsized":"2023-03-03T06:25:18.335118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:33:19.697671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350575+0000","last_change":"2023-03-03T06:25:13.308653+0000","last_active":"2023-03-03T06:25:18.350575+0000","last_peered":"2023-03-03T06:25:18.350575+0000","last_clean":"2023-03-03T06:25:18.350575+0000","last_became_active":"2023-03-03T06:25:13.308404+0000","last_became_peered":"2023-03-03T06:25:13.308404+0000","last_unstale":"2023-03-03T06:25:18.350575+0000","last_undegraded":"2023-03-03T06:25:18.350575+0000","last_fullsized":"2023-03-03T06:25:18.350575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:42:29.976451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.217056+0000","last_change":"2023-03-03T06:25:18.369545+0000","last_active":"2023-03-03T06:25:19.217056+0000","last_peered":"2023-03-03T06:25:19.217056+0000","last_clean":"2023-03-03T06:25:19.217056+0000","last_became_active":"2023-03-03T06:25:18.369075+0000","last_became_peered":"2023-03-03T06:25:18.369075+0000","last_unstale":"2023-03-03T06:25:19.217056+0000","last_undegraded":"2023-03-03T06:25:19.217056+0000","last_fullsized":"2023-03-03T06:25:19.217056+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:45:30.096214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352873+0000","last_change":"2023-03-03T06:25:13.836258+0000","last_active":"2023-03-03T06:25:18.352873+0000","last_peered":"2023-03-03T06:25:18.352873+0000","last_clean":"2023-03-03T06:25:18.352873+0000","last_became_active":"2023-03-03T06:25:13.835993+0000","last_became_peered":"2023-03-03T06:25:13.835993+0000","last_unstale":"2023-03-03T06:25:18.352873+0000","last_undegraded":"2023-03-03T06:25:18.352873+0000","last_fullsized":"2023-03-03T06:25:18.352873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.350886+0000","last_change":"2023-03-03T06:25:13.307887+0000","last_active":"2023-03-03T06:25:18.350886+0000","last_peered":"2023-03-03T06:25:18.350886+0000","last_clean":"2023-03-03T06:25:18.350886+0000","last_became_active":"2023-03-03T06:25:13.307622+0000","last_became_peered":"2023-03-03T06:25:13.307622+0000","last_unstale":"2023-03-03T06:25:18.350886+0000","last_undegraded":"2023-03-03T06:25:18.350886+0000","last_fullsized":"2023-03-03T06:25:18.350886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:51:44.342844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.462959+0000","last_change":"2023-03-03T06:25:18.366887+0000","last_active":"2023-03-03T06:25:19.462959+0000","last_peered":"2023-03-03T06:25:19.462959+0000","last_clean":"2023-03-03T06:25:19.462959+0000","last_became_active":"2023-03-03T06:25:18.366248+0000","last_became_peered":"2023-03-03T06:25:18.366248+0000","last_unstale":"2023-03-03T06:25:19.462959+0000","last_undegraded":"2023-03-03T06:25:19.462959+0000","last_fullsized":"2023-03-03T06:25:19.462959+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:06:08.404601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879274+0000","last_change":"2023-03-03T06:25:18.374976+0000","last_active":"2023-03-03T06:25:18.879274+0000","last_peered":"2023-03-03T06:25:18.879274+0000","last_clean":"2023-03-03T06:25:18.879274+0000","last_became_active":"2023-03-03T06:25:18.374414+0000","last_became_peered":"2023-03-03T06:25:18.374414+0000","last_unstale":"2023-03-03T06:25:18.879274+0000","last_undegraded":"2023-03-03T06:25:18.879274+0000","last_fullsized":"2023-03-03T06:25:18.879274+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:52:08.870718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.340487+0000","last_change":"2023-03-03T06:25:13.297905+0000","last_active":"2023-03-03T06:25:18.340487+0000","last_peered":"2023-03-03T06:25:18.340487+0000","last_clean":"2023-03-03T06:25:18.340487+0000","last_became_active":"2023-03-03T06:25:13.297568+0000","last_became_peered":"2023-03-03T06:25:13.297568+0000","last_unstale":"2023-03-03T06:25:18.340487+0000","last_undegraded":"2023-03-03T06:25:18.340487+0000","last_fullsized":"2023-03-03T06:25:18.340487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:17:34.388274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343217+0000","last_change":"2023-03-03T06:25:13.851715+0000","last_active":"2023-03-03T06:25:18.343217+0000","last_peered":"2023-03-03T06:25:18.343217+0000","last_clean":"2023-03-03T06:25:18.343217+0000","last_became_active":"2023-03-03T06:25:13.851377+0000","last_became_peered":"2023-03-03T06:25:13.851377+0000","last_unstale":"2023-03-03T06:25:18.343217+0000","last_undegraded":"2023-03-03T06:25:18.343217+0000","last_fullsized":"2023-03-03T06:25:18.343217+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467810+0000","last_change":"2023-03-03T06:25:18.356507+0000","last_active":"2023-03-03T06:25:19.467810+0000","last_peered":"2023-03-03T06:25:19.467810+0000","last_clean":"2023-03-03T06:25:19.467810+0000","last_became_active":"2023-03-03T06:25:18.356097+0000","last_became_peered":"2023-03-03T06:25:18.356097+0000","last_unstale":"2023-03-03T06:25:19.467810+0000","last_undegraded":"2023-03-03T06:25:19.467810+0000","last_fullsized":"2023-03-03T06:25:19.467810+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:59:33.700378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.351468+0000","last_change":"2023-03-03T06:25:13.306049+0000","last_active":"2023-03-03T06:25:18.351468+0000","last_peered":"2023-03-03T06:25:18.351468+0000","last_clean":"2023-03-03T06:25:18.351468+0000","last_became_active":"2023-03-03T06:25:13.305742+0000","last_became_peered":"2023-03-03T06:25:13.305742+0000","last_unstale":"2023-03-03T06:25:18.351468+0000","last_undegraded":"2023-03-03T06:25:18.351468+0000","last_fullsized":"2023-03-03T06:25:18.351468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:29:05.784203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344608+0000","last_change":"2023-03-03T06:25:13.841823+0000","last_active":"2023-03-03T06:25:18.344608+0000","last_peered":"2023-03-03T06:25:18.344608+0000","last_clean":"2023-03-03T06:25:18.344608+0000","last_became_active":"2023-03-03T06:25:13.841581+0000","last_became_peered":"2023-03-03T06:25:13.841581+0000","last_unstale":"2023-03-03T06:25:18.344608+0000","last_undegraded":"2023-03-03T06:25:18.344608+0000","last_fullsized":"2023-03-03T06:25:18.344608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:08:27.149181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342620+0000","last_change":"2023-03-03T06:25:13.307869+0000","last_active":"2023-03-03T06:25:18.342620+0000","last_peered":"2023-03-03T06:25:18.342620+0000","last_clean":"2023-03-03T06:25:18.342620+0000","last_became_active":"2023-03-03T06:25:13.303371+0000","last_became_peered":"2023-03-03T06:25:13.303371+0000","last_unstale":"2023-03-03T06:25:18.342620+0000","last_undegraded":"2023-03-03T06:25:18.342620+0000","last_fullsized":"2023-03-03T06:25:18.342620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:38.035008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352774+0000","last_change":"2023-03-03T06:25:13.853988+0000","last_active":"2023-03-03T06:25:18.352774+0000","last_peered":"2023-03-03T06:25:18.352774+0000","last_clean":"2023-03-03T06:25:18.352774+0000","last_became_active":"2023-03-03T06:25:13.853714+0000","last_became_peered":"2023-03-03T06:25:13.853714+0000","last_unstale":"2023-03-03T06:25:18.352774+0000","last_undegraded":"2023-03-03T06:25:18.352774+0000","last_fullsized":"2023-03-03T06:25:18.352774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:51:30.984078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:19.467763+0000","last_change":"2023-03-03T06:25:18.372232+0000","last_active":"2023-03-03T06:25:19.467763+0000","last_peered":"2023-03-03T06:25:19.467763+0000","last_clean":"2023-03-03T06:25:19.467763+0000","last_became_active":"2023-03-03T06:25:18.371809+0000","last_became_peered":"2023-03-03T06:25:18.371809+0000","last_unstale":"2023-03-03T06:25:19.467763+0000","last_undegraded":"2023-03-03T06:25:19.467763+0000","last_fullsized":"2023-03-03T06:25:19.467763+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:11:23.940316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346535+0000","last_change":"2023-03-03T06:25:13.307069+0000","last_active":"2023-03-03T06:25:18.346535+0000","last_peered":"2023-03-03T06:25:18.346535+0000","last_clean":"2023-03-03T06:25:18.346535+0000","last_became_active":"2023-03-03T06:25:13.306791+0000","last_became_peered":"2023-03-03T06:25:13.306791+0000","last_unstale":"2023-03-03T06:25:18.346535+0000","last_undegraded":"2023-03-03T06:25:18.346535+0000","last_fullsized":"2023-03-03T06:25:18.346535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:12.236400+0000","last_clean_scrub_stamp":"2023-03-03T06:25:12.236400+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:19:48.531435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343519+0000","last_change":"2023-03-03T06:25:13.830026+0000","last_active":"2023-03-03T06:25:18.343519+0000","last_peered":"2023-03-03T06:25:18.343519+0000","last_clean":"2023-03-03T06:25:18.343519+0000","last_became_active":"2023-03-03T06:25:13.829773+0000","last_became_peered":"2023-03-03T06:25:13.829773+0000","last_unstale":"2023-03-03T06:25:18.343519+0000","last_undegraded":"2023-03-03T06:25:18.343519+0000","last_fullsized":"2023-03-03T06:25:18.343519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:35:57.127264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.879167+0000","last_change":"2023-03-03T06:25:18.374930+0000","last_active":"2023-03-03T06:25:18.879167+0000","last_peered":"2023-03-03T06:25:18.879167+0000","last_clean":"2023-03-03T06:25:18.879167+0000","last_became_active":"2023-03-03T06:25:18.374374+0000","last_became_peered":"2023-03-03T06:25:18.374374+0000","last_unstale":"2023-03-03T06:25:18.879167+0000","last_undegraded":"2023-03-03T06:25:18.879167+0000","last_fullsized":"2023-03-03T06:25:18.879167+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:17.299063+0000","last_clean_scrub_stamp":"2023-03-03T06:25:17.299063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:09:37.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.340359+0000","last_change":"2023-03-03T06:25:13.840311+0000","last_active":"2023-03-03T06:25:18.340359+0000","last_peered":"2023-03-03T06:25:18.340359+0000","last_clean":"2023-03-03T06:25:18.340359+0000","last_became_active":"2023-03-03T06:25:13.839648+0000","last_became_peered":"2023-03-03T06:25:13.839648+0000","last_unstale":"2023-03-03T06:25:18.340359+0000","last_undegraded":"2023-03-03T06:25:18.340359+0000","last_fullsized":"2023-03-03T06:25:18.340359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:29:06.800310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337721+0000","last_change":"2023-03-03T06:25:13.836213+0000","last_active":"2023-03-03T06:25:18.337721+0000","last_peered":"2023-03-03T06:25:18.337721+0000","last_clean":"2023-03-03T06:25:18.337721+0000","last_became_active":"2023-03-03T06:25:13.835890+0000","last_became_peered":"2023-03-03T06:25:13.835890+0000","last_unstale":"2023-03-03T06:25:18.337721+0000","last_undegraded":"2023-03-03T06:25:18.337721+0000","last_fullsized":"2023-03-03T06:25:18.337721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:55.025383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335018+0000","last_change":"2023-03-03T06:25:13.834786+0000","last_active":"2023-03-03T06:25:18.335018+0000","last_peered":"2023-03-03T06:25:18.335018+0000","last_clean":"2023-03-03T06:25:18.335018+0000","last_became_active":"2023-03-03T06:25:13.834531+0000","last_became_peered":"2023-03-03T06:25:13.834531+0000","last_unstale":"2023-03-03T06:25:18.335018+0000","last_undegraded":"2023-03-03T06:25:18.335018+0000","last_fullsized":"2023-03-03T06:25:18.335018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:07:17.177238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342130+0000","last_change":"2023-03-03T06:25:13.850334+0000","last_active":"2023-03-03T06:25:18.342130+0000","last_peered":"2023-03-03T06:25:18.342130+0000","last_clean":"2023-03-03T06:25:18.342130+0000","last_became_active":"2023-03-03T06:25:13.850082+0000","last_became_peered":"2023-03-03T06:25:13.850082+0000","last_unstale":"2023-03-03T06:25:18.342130+0000","last_undegraded":"2023-03-03T06:25:18.342130+0000","last_fullsized":"2023-03-03T06:25:18.342130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357706+0000","last_change":"2023-03-03T06:25:13.850690+0000","last_active":"2023-03-03T06:25:18.357706+0000","last_peered":"2023-03-03T06:25:18.357706+0000","last_clean":"2023-03-03T06:25:18.357706+0000","last_became_active":"2023-03-03T06:25:13.850392+0000","last_became_peered":"2023-03-03T06:25:13.850392+0000","last_unstale":"2023-03-03T06:25:18.357706+0000","last_undegraded":"2023-03-03T06:25:18.357706+0000","last_fullsized":"2023-03-03T06:25:18.357706+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352673+0000","last_change":"2023-03-03T06:25:13.853965+0000","last_active":"2023-03-03T06:25:18.352673+0000","last_peered":"2023-03-03T06:25:18.352673+0000","last_clean":"2023-03-03T06:25:18.352673+0000","last_became_active":"2023-03-03T06:25:13.853689+0000","last_became_peered":"2023-03-03T06:25:13.853689+0000","last_unstale":"2023-03-03T06:25:18.352673+0000","last_undegraded":"2023-03-03T06:25:18.352673+0000","last_fullsized":"2023-03-03T06:25:18.352673+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:09:17.786520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.357579+0000","last_change":"2023-03-03T06:25:13.828945+0000","last_active":"2023-03-03T06:25:18.357579+0000","last_peered":"2023-03-03T06:25:18.357579+0000","last_clean":"2023-03-03T06:25:18.357579+0000","last_became_active":"2023-03-03T06:25:13.828655+0000","last_became_peered":"2023-03-03T06:25:13.828655+0000","last_unstale":"2023-03-03T06:25:18.357579+0000","last_undegraded":"2023-03-03T06:25:18.357579+0000","last_fullsized":"2023-03-03T06:25:18.357579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:51:30.984078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.358304+0000","last_change":"2023-03-03T06:25:13.834307+0000","last_active":"2023-03-03T06:25:18.358304+0000","last_peered":"2023-03-03T06:25:18.358304+0000","last_clean":"2023-03-03T06:25:18.358304+0000","last_became_active":"2023-03-03T06:25:13.834054+0000","last_became_peered":"2023-03-03T06:25:13.834054+0000","last_unstale":"2023-03-03T06:25:18.358304+0000","last_undegraded":"2023-03-03T06:25:18.358304+0000","last_fullsized":"2023-03-03T06:25:18.358304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:54:02.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346796+0000","last_change":"2023-03-03T06:25:13.856830+0000","last_active":"2023-03-03T06:25:18.346796+0000","last_peered":"2023-03-03T06:25:18.346796+0000","last_clean":"2023-03-03T06:25:18.346796+0000","last_became_active":"2023-03-03T06:25:13.856494+0000","last_became_peered":"2023-03-03T06:25:13.856494+0000","last_unstale":"2023-03-03T06:25:18.346796+0000","last_undegraded":"2023-03-03T06:25:18.346796+0000","last_fullsized":"2023-03-03T06:25:18.346796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:59:50.232210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337609+0000","last_change":"2023-03-03T06:25:13.836186+0000","last_active":"2023-03-03T06:25:18.337609+0000","last_peered":"2023-03-03T06:25:18.337609+0000","last_clean":"2023-03-03T06:25:18.337609+0000","last_became_active":"2023-03-03T06:25:13.835481+0000","last_became_peered":"2023-03-03T06:25:13.835481+0000","last_unstale":"2023-03-03T06:25:18.337609+0000","last_undegraded":"2023-03-03T06:25:18.337609+0000","last_fullsized":"2023-03-03T06:25:18.337609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:30:23.964335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.342530+0000","last_change":"2023-03-03T06:25:13.834687+0000","last_active":"2023-03-03T06:25:18.342530+0000","last_peered":"2023-03-03T06:25:18.342530+0000","last_clean":"2023-03-03T06:25:18.342530+0000","last_became_active":"2023-03-03T06:25:13.834424+0000","last_became_peered":"2023-03-03T06:25:13.834424+0000","last_unstale":"2023-03-03T06:25:18.342530+0000","last_undegraded":"2023-03-03T06:25:18.342530+0000","last_fullsized":"2023-03-03T06:25:18.342530+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.346438+0000","last_change":"2023-03-03T06:25:13.855849+0000","last_active":"2023-03-03T06:25:18.346438+0000","last_peered":"2023-03-03T06:25:18.346438+0000","last_clean":"2023-03-03T06:25:18.346438+0000","last_became_active":"2023-03-03T06:25:13.855609+0000","last_became_peered":"2023-03-03T06:25:13.855609+0000","last_unstale":"2023-03-03T06:25:18.346438+0000","last_undegraded":"2023-03-03T06:25:18.346438+0000","last_fullsized":"2023-03-03T06:25:18.346438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:08:40.068106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335348+0000","last_change":"2023-03-03T06:25:13.846670+0000","last_active":"2023-03-03T06:25:18.335348+0000","last_peered":"2023-03-03T06:25:18.335348+0000","last_clean":"2023-03-03T06:25:18.335348+0000","last_became_active":"2023-03-03T06:25:13.846349+0000","last_became_peered":"2023-03-03T06:25:13.846349+0000","last_unstale":"2023-03-03T06:25:18.335348+0000","last_undegraded":"2023-03-03T06:25:18.335348+0000","last_fullsized":"2023-03-03T06:25:18.335348+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:30:23.964335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337310+0000","last_change":"2023-03-03T06:25:13.839483+0000","last_active":"2023-03-03T06:25:18.337310+0000","last_peered":"2023-03-03T06:25:18.337310+0000","last_clean":"2023-03-03T06:25:18.337310+0000","last_became_active":"2023-03-03T06:25:13.838497+0000","last_became_peered":"2023-03-03T06:25:13.838497+0000","last_unstale":"2023-03-03T06:25:18.337310+0000","last_undegraded":"2023-03-03T06:25:18.337310+0000","last_fullsized":"2023-03-03T06:25:18.337310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:41:08.122967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334979+0000","last_change":"2023-03-03T06:25:13.855262+0000","last_active":"2023-03-03T06:25:18.334979+0000","last_peered":"2023-03-03T06:25:18.334979+0000","last_clean":"2023-03-03T06:25:18.334979+0000","last_became_active":"2023-03-03T06:25:13.848660+0000","last_became_peered":"2023-03-03T06:25:13.848660+0000","last_unstale":"2023-03-03T06:25:18.334979+0000","last_undegraded":"2023-03-03T06:25:18.334979+0000","last_fullsized":"2023-03-03T06:25:18.334979+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:14:55.025383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344570+0000","last_change":"2023-03-03T06:25:13.842845+0000","last_active":"2023-03-03T06:25:18.344570+0000","last_peered":"2023-03-03T06:25:18.344570+0000","last_clean":"2023-03-03T06:25:18.344570+0000","last_became_active":"2023-03-03T06:25:13.842580+0000","last_became_peered":"2023-03-03T06:25:13.842580+0000","last_unstale":"2023-03-03T06:25:18.344570+0000","last_undegraded":"2023-03-03T06:25:18.344570+0000","last_fullsized":"2023-03-03T06:25:18.344570+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:28:42.514217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335348+0000","last_change":"2023-03-03T06:25:13.836306+0000","last_active":"2023-03-03T06:25:18.335348+0000","last_peered":"2023-03-03T06:25:18.335348+0000","last_clean":"2023-03-03T06:25:18.335348+0000","last_became_active":"2023-03-03T06:25:13.835953+0000","last_became_peered":"2023-03-03T06:25:13.835953+0000","last_unstale":"2023-03-03T06:25:18.335348+0000","last_undegraded":"2023-03-03T06:25:18.335348+0000","last_fullsized":"2023-03-03T06:25:18.335348+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:35:57.127264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.351349+0000","last_change":"2023-03-03T06:25:13.854132+0000","last_active":"2023-03-03T06:25:18.351349+0000","last_peered":"2023-03-03T06:25:18.351349+0000","last_clean":"2023-03-03T06:25:18.351349+0000","last_became_active":"2023-03-03T06:25:13.853786+0000","last_became_peered":"2023-03-03T06:25:13.853786+0000","last_unstale":"2023-03-03T06:25:18.351349+0000","last_undegraded":"2023-03-03T06:25:18.351349+0000","last_fullsized":"2023-03-03T06:25:18.351349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353001+0000","last_change":"2023-03-03T06:25:13.835134+0000","last_active":"2023-03-03T06:25:18.353001+0000","last_peered":"2023-03-03T06:25:18.353001+0000","last_clean":"2023-03-03T06:25:18.353001+0000","last_became_active":"2023-03-03T06:25:13.834844+0000","last_became_peered":"2023-03-03T06:25:13.834844+0000","last_unstale":"2023-03-03T06:25:18.353001+0000","last_undegraded":"2023-03-03T06:25:18.353001+0000","last_fullsized":"2023-03-03T06:25:18.353001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.337727+0000","last_change":"2023-03-03T06:25:13.830031+0000","last_active":"2023-03-03T06:25:18.337727+0000","last_peered":"2023-03-03T06:25:18.337727+0000","last_clean":"2023-03-03T06:25:18.337727+0000","last_became_active":"2023-03-03T06:25:13.829770+0000","last_became_peered":"2023-03-03T06:25:13.829770+0000","last_unstale":"2023-03-03T06:25:18.337727+0000","last_undegraded":"2023-03-03T06:25:18.337727+0000","last_fullsized":"2023-03-03T06:25:18.337727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:13:06.900541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.336311+0000","last_change":"2023-03-03T06:25:13.858248+0000","last_active":"2023-03-03T06:25:18.336311+0000","last_peered":"2023-03-03T06:25:18.336311+0000","last_clean":"2023-03-03T06:25:18.336311+0000","last_became_active":"2023-03-03T06:25:13.857884+0000","last_became_peered":"2023-03-03T06:25:13.857884+0000","last_unstale":"2023-03-03T06:25:18.336311+0000","last_undegraded":"2023-03-03T06:25:18.336311+0000","last_fullsized":"2023-03-03T06:25:18.336311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:32:39.051115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334811+0000","last_change":"2023-03-03T06:25:13.847616+0000","last_active":"2023-03-03T06:25:18.334811+0000","last_peered":"2023-03-03T06:25:18.334811+0000","last_clean":"2023-03-03T06:25:18.334811+0000","last_became_active":"2023-03-03T06:25:13.847110+0000","last_became_peered":"2023-03-03T06:25:13.847110+0000","last_unstale":"2023-03-03T06:25:18.334811+0000","last_undegraded":"2023-03-03T06:25:18.334811+0000","last_fullsized":"2023-03-03T06:25:18.334811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:52:13.551129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353150+0000","last_change":"2023-03-03T06:25:13.851731+0000","last_active":"2023-03-03T06:25:18.353150+0000","last_peered":"2023-03-03T06:25:18.353150+0000","last_clean":"2023-03-03T06:25:18.353150+0000","last_became_active":"2023-03-03T06:25:13.849868+0000","last_became_peered":"2023-03-03T06:25:13.849868+0000","last_unstale":"2023-03-03T06:25:18.353150+0000","last_undegraded":"2023-03-03T06:25:18.353150+0000","last_fullsized":"2023-03-03T06:25:18.353150+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:15:09.286575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.343321+0000","last_change":"2023-03-03T06:25:13.846247+0000","last_active":"2023-03-03T06:25:18.343321+0000","last_peered":"2023-03-03T06:25:18.343321+0000","last_clean":"2023-03-03T06:25:18.343321+0000","last_became_active":"2023-03-03T06:25:13.845983+0000","last_became_peered":"2023-03-03T06:25:13.845983+0000","last_unstale":"2023-03-03T06:25:18.343321+0000","last_undegraded":"2023-03-03T06:25:18.343321+0000","last_fullsized":"2023-03-03T06:25:18.343321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:54:02.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.352766+0000","last_change":"2023-03-03T06:25:13.858013+0000","last_active":"2023-03-03T06:25:18.352766+0000","last_peered":"2023-03-03T06:25:18.352766+0000","last_clean":"2023-03-03T06:25:18.352766+0000","last_became_active":"2023-03-03T06:25:13.857706+0000","last_became_peered":"2023-03-03T06:25:13.857706+0000","last_unstale":"2023-03-03T06:25:18.352766+0000","last_undegraded":"2023-03-03T06:25:18.352766+0000","last_fullsized":"2023-03-03T06:25:18.352766+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:54:02.649490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.351182+0000","last_change":"2023-03-03T06:25:13.854691+0000","last_active":"2023-03-03T06:25:18.351182+0000","last_peered":"2023-03-03T06:25:18.351182+0000","last_clean":"2023-03-03T06:25:18.351182+0000","last_became_active":"2023-03-03T06:25:13.854332+0000","last_became_peered":"2023-03-03T06:25:13.854332+0000","last_unstale":"2023-03-03T06:25:18.351182+0000","last_undegraded":"2023-03-03T06:25:18.351182+0000","last_fullsized":"2023-03-03T06:25:18.351182+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:01:14.425412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.334771+0000","last_change":"2023-03-03T06:25:13.836791+0000","last_active":"2023-03-03T06:25:18.334771+0000","last_peered":"2023-03-03T06:25:18.334771+0000","last_clean":"2023-03-03T06:25:18.334771+0000","last_became_active":"2023-03-03T06:25:13.836500+0000","last_became_peered":"2023-03-03T06:25:13.836500+0000","last_unstale":"2023-03-03T06:25:18.334771+0000","last_undegraded":"2023-03-03T06:25:18.334771+0000","last_fullsized":"2023-03-03T06:25:18.334771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:38:47.103593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.344030+0000","last_change":"2023-03-03T06:25:13.846407+0000","last_active":"2023-03-03T06:25:18.344030+0000","last_peered":"2023-03-03T06:25:18.344030+0000","last_clean":"2023-03-03T06:25:18.344030+0000","last_became_active":"2023-03-03T06:25:13.846119+0000","last_became_peered":"2023-03-03T06:25:13.846119+0000","last_unstale":"2023-03-03T06:25:18.344030+0000","last_undegraded":"2023-03-03T06:25:18.344030+0000","last_fullsized":"2023-03-03T06:25:18.344030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.341377+0000","last_change":"2023-03-03T06:25:13.843673+0000","last_active":"2023-03-03T06:25:18.341377+0000","last_peered":"2023-03-03T06:25:18.341377+0000","last_clean":"2023-03-03T06:25:18.341377+0000","last_became_active":"2023-03-03T06:25:13.843408+0000","last_became_peered":"2023-03-03T06:25:13.843408+0000","last_unstale":"2023-03-03T06:25:18.341377+0000","last_undegraded":"2023-03-03T06:25:18.341377+0000","last_fullsized":"2023-03-03T06:25:18.341377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:51:45.358951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.335571+0000","last_change":"2023-03-03T06:25:13.847791+0000","last_active":"2023-03-03T06:25:18.335571+0000","last_peered":"2023-03-03T06:25:18.335571+0000","last_clean":"2023-03-03T06:25:18.335571+0000","last_became_active":"2023-03-03T06:25:13.847378+0000","last_became_peered":"2023-03-03T06:25:13.847378+0000","last_unstale":"2023-03-03T06:25:18.335571+0000","last_undegraded":"2023-03-03T06:25:18.335571+0000","last_fullsized":"2023-03-03T06:25:18.335571+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:05:53.754239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.353098+0000","last_change":"2023-03-03T06:25:13.835727+0000","last_active":"2023-03-03T06:25:18.353098+0000","last_peered":"2023-03-03T06:25:18.353098+0000","last_clean":"2023-03-03T06:25:18.353098+0000","last_became_active":"2023-03-03T06:25:13.835462+0000","last_became_peered":"2023-03-03T06:25:13.835462+0000","last_unstale":"2023-03-03T06:25:18.353098+0000","last_undegraded":"2023-03-03T06:25:18.353098+0000","last_fullsized":"2023-03-03T06:25:18.353098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:17:48.628960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-03T06:25:18.336596+0000","last_change":"2023-03-03T06:25:13.827811+0000","last_active":"2023-03-03T06:25:18.336596+0000","last_peered":"2023-03-03T06:25:18.336596+0000","last_clean":"2023-03-03T06:25:18.336596+0000","last_became_active":"2023-03-03T06:25:13.827532+0000","last_became_peered":"2023-03-03T06:25:13.827532+0000","last_unstale":"2023-03-03T06:25:18.336596+0000","last_undegraded":"2023-03-03T06:25:18.336596+0000","last_fullsized":"2023-03-03T06:25:18.336596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:13.252507+0000","last_clean_scrub_stamp":"2023-03-03T06:25:13.252507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:25:09.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4527,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13456,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":12544,"kb_avail":94358384,"statfs":{"total":96636764160,"available":96622985216,"internally_reserved":0,"allocated":925696,"data_stored":718901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12845056},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3939999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.508}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6439999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12272,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":11968,"kb_avail":94359568,"statfs":{"total":96636764160,"available":96624197632,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12255232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7070000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6279999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6040000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12280,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":11968,"kb_avail":94359560,"statfs":{"total":96636764160,"available":96624189440,"internally_reserved":0,"allocated":311296,"data_stored":125208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12255232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6040000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.544}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6739999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12356,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":12032,"kb_avail":94359484,"statfs":{"total":96636764160,"available":96624111616,"internally_reserved":0,"allocated":323584,"data_stored":128373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12320768},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4870000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.843}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7749999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2080000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1520000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8112,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363728,"statfs":{"total":96636764160,"available":96628457472,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8240,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363600,"statfs":{"total":96636764160,"available":96628326400,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9272,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362568,"statfs":{"total":96636764160,"available":96627269632,"internally_reserved":0,"allocated":901120,"data_stored":715506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8620000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7050000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.758}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-03T06:25:49.241 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-03-03T06:25:49.241 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-03-03T06:25:49.241 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-03-03T06:25:49.241 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-03-03T06:25:49.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.475+0000 7f94c21d2700 1 Processor -- start 2023-03-03T06:25:49.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.475+0000 7f94c21d2700 1 -- start start 2023-03-03T06:25:49.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.475+0000 7f94c21d2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94bc13fb50 0x7f94bc15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.475+0000 7f94c21d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc15b5e0 0x7f94bc070c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94c21d2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94bc0cc240 con 0x7f94bc15acd0 2023-03-03T06:25:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94c21d2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94bc0ce830 con 0x7f94bc13fb50 2023-03-03T06:25:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94c21d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc140080 con 0x7f94bc15b5e0 2023-03-03T06:25:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94c11d0700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f94bc15acd0 0x7f94bc0b0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33792/0 (socket says 172.21.15.3:33792) 2023-03-03T06:25:49.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94c11d0700 1 -- 172.21.15.3:0/2120531590 learned_addr learned my addr 172.21.15.3:0/2120531590 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:49.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94c09cf700 1 --2- 172.21.15.3:0/2120531590 >> v2:172.21.15.32:3300/0 conn(0x7f94bc13fb50 0x7f94bc15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:49.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94c19d1700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc15b5e0 0x7f94bc070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:49.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869112659 0 0) 0x7f94bc0cc240 con 0x7f94bc15acd0 2023-03-03T06:25:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a80035d0 con 0x7f94bc15acd0 2023-03-03T06:25:49.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3424759811 0 0) 0x7f94a80035d0 con 0x7f94bc15acd0 2023-03-03T06:25:49.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc15b5e0 msgr2=0x7f94bc070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:49.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc15b5e0 0x7f94bc070c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 >> v2:172.21.15.32:3300/0 conn(0x7f94bc13fb50 msgr2=0x7f94bc15b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:49.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 --2- 172.21.15.3:0/2120531590 >> v2:172.21.15.32:3300/0 conn(0x7f94bc13fb50 0x7f94bc15b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc140360 con 0x7f94bc15acd0 2023-03-03T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f94b80019f0 con 0x7f94bc15acd0 2023-03-03T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94c19d1700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc15b5e0 0x7f94bc070c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 >> v1:172.21.15.3:6789/0 conn(0x7f94bc15acd0 legacy=0x7f94bc0b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8003a20 0x7f94a8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a8004400 0x7f94a80048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a8004f40 0x7f94a800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc0ce830 con 0x7f94a8004400 2023-03-03T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc140080 con 0x7f94a8004f40 2023-03-03T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a800bf70 con 0x7f94a8003a20 2023-03-03T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94c09cf700 1 --2- 172.21.15.3:0/2120531590 >> v2:172.21.15.32:3300/0 conn(0x7f94bc13fb50 0x7f94bc15b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94c11d0700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8003a20 0x7f94a8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94c19d1700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a8004f40 0x7f94a800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.479+0000 7f94c09cf700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a8004400 0x7f94a80048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c11d0700 1 -- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a8004f40 msgr2=0x7f94a800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c11d0700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a8004f40 0x7f94a800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c11d0700 1 -- 172.21.15.3:0/2120531590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a8004400 msgr2=0x7f94a80048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c11d0700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a8004400 0x7f94a80048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c11d0700 1 -- 172.21.15.3:0/2120531590 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a80035d0 con 0x7f94a8003a20 2023-03-03T06:25:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c19d1700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a8004f40 0x7f94a800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c11d0700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8003a20 0x7f94a8003e30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f94b800a860 tx=0x7f94a80058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d6185208585a26 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8011760 con 0x7f94a8003a20 2023-03-03T06:25:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b80118e0 con 0x7f94a8003a20 2023-03-03T06:25:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94b3fff700 1 -- 172.21.15.3:0/2120531590 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8011c90 con 0x7f94a8003a20 2023-03-03T06:25:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c21d2700 1 -- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8003a20 msgr2=0x7f94a8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c21d2700 1 --2- 172.21.15.3:0/2120531590 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8003a20 0x7f94a8003e30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f94b800a860 tx=0x7f94a80058a0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c19d1700 1 -- 172.21.15.3:0/2120531590 reap_dead start 2023-03-03T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c21d2700 1 -- 172.21.15.3:0/2120531590 shutdown_connections 2023-03-03T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c21d2700 1 -- 172.21.15.3:0/2120531590 >> 172.21.15.3:0/2120531590 conn(0x7f94bc06a160 msgr2=0x7f94bc0607c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c21d2700 1 -- 172.21.15.3:0/2120531590 shutdown_connections 2023-03-03T06:25:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.483+0000 7f94c21d2700 1 -- 172.21.15.3:0/2120531590 wait complete. 2023-03-03T06:25:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.487+0000 7f94c21d2700 1 Processor -- start 2023-03-03T06:25:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.487+0000 7f94c21d2700 1 -- start start 2023-03-03T06:25:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c21d2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a8003a20 0x7f94bc13e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c21d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc06cbb0 0x7f94bc1f9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c21d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc06cfe0 0x7f94bc1fa230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c21d2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc13abe0 con 0x7f94a8003a20 2023-03-03T06:25:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c21d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc13ae30 con 0x7f94bc06cbb0 2023-03-03T06:25:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c21d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc13b110 con 0x7f94bc06cfe0 2023-03-03T06:25:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c11d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a8003a20 0x7f94bc13e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c19d1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc06cfe0 0x7f94bc1fa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c19d1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc06cfe0 0x7f94bc1fa230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:55658/0 (socket says 172.21.15.3:55658) 2023-03-03T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c19d1700 1 -- 172.21.15.3:0/4193211516 learned_addr learned my addr 172.21.15.3:0/4193211516 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c09cf700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc06cbb0 0x7f94bc1f9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c19d1700 1 -- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc06cbb0 msgr2=0x7f94bc1f9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c19d1700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc06cbb0 0x7f94bc1f9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c19d1700 1 -- 172.21.15.3:0/4193211516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a8003a20 msgr2=0x7f94bc13e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c19d1700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a8003a20 0x7f94bc13e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c19d1700 1 -- 172.21.15.3:0/4193211516 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc140920 con 0x7f94bc06cfe0 2023-03-03T06:25:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c11d0700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a8003a20 0x7f94bc13e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94c19d1700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc06cfe0 0x7f94bc1fa230 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0cb390 tx=0x7f94b4011eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.491+0000 7f94b1ffb700 1 -- 172.21.15.3:0/4193211516 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b400f8f0 con 0x7f94bc06cfe0 2023-03-03T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.495+0000 7f94b1ffb700 1 -- 172.21.15.3:0/4193211516 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b400fa70 con 0x7f94bc06cfe0 2023-03-03T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.495+0000 7f94b1ffb700 1 -- 172.21.15.3:0/4193211516 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b400fe20 con 0x7f94bc06cfe0 2023-03-03T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.495+0000 7f94c21d2700 1 -- 172.21.15.3:0/4193211516 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc140640 con 0x7f94bc06cfe0 2023-03-03T06:25:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.495+0000 7f94c21d2700 1 -- 172.21.15.3:0/4193211516 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc0b0910 con 0x7f94bc06cfe0 2023-03-03T06:25:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.495+0000 7f94c21d2700 1 -- 172.21.15.3:0/4193211516 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94bc06a560 con 0x7f94bc06cfe0 2023-03-03T06:25:49.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.499+0000 7f94b1ffb700 1 -- 172.21.15.3:0/4193211516 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f94b403baf0 con 0x7f94bc06cfe0 2023-03-03T06:25:49.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.503+0000 7f94b1ffb700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f94a0064f10 0x7f94a00673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.503+0000 7f94b1ffb700 1 -- 172.21.15.3:0/4193211516 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f94b4082490 con 0x7f94bc06cfe0 2023-03-03T06:25:49.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.503+0000 7f94c11d0700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f94a0064f10 0x7f94a00673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:49.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.503+0000 7f94c11d0700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f94a0064f10 0x7f94a00673c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f94b8000f80 tx=0x7f94b801d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:49.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.507+0000 7f94b1ffb700 1 -- 172.21.15.3:0/4193211516 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f94b40502d0 con 0x7f94bc06cfe0 2023-03-03T06:25:49.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.719+0000 7f94c21d2700 1 -- 172.21.15.3:0/4193211516 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f94bc0cbcd0 con 0x7f94bc06cfe0 2023-03-03T06:25:49.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.723+0000 7f94b1ffb700 1 -- 172.21.15.3:0/4193211516 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f94b40538f0 con 0x7f94bc06cfe0 2023-03-03T06:25:49.726 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:49.726 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-03-03T06:25:49.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.727+0000 7f94c21d2700 1 -- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f94a0064f10 msgr2=0x7f94a00673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:49.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.727+0000 7f94c21d2700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f94a0064f10 0x7f94a00673c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f94b8000f80 tx=0x7f94b801d040 comp rx=0 tx=0).stop 2023-03-03T06:25:49.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.727+0000 7f94c21d2700 1 -- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc06cfe0 msgr2=0x7f94bc1fa230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:49.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.727+0000 7f94c21d2700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc06cfe0 0x7f94bc1fa230 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0cb390 tx=0x7f94b4011eb0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.727+0000 7f94c21d2700 1 -- 172.21.15.3:0/4193211516 shutdown_connections 2023-03-03T06:25:49.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.727+0000 7f94c21d2700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f94a0064f10 0x7f94a00673c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.727+0000 7f94c21d2700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94bc06cfe0 0x7f94bc1fa230 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.727+0000 7f94c21d2700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc06cbb0 0x7f94bc1f9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.727+0000 7f94c21d2700 1 --2- 172.21.15.3:0/4193211516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a8003a20 0x7f94bc13e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.727+0000 7f94c21d2700 1 -- 172.21.15.3:0/4193211516 >> 172.21.15.3:0/4193211516 conn(0x7f94bc06a160 msgr2=0x7f94bc0607c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:49.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.727+0000 7f94c21d2700 1 -- 172.21.15.3:0/4193211516 shutdown_connections 2023-03-03T06:25:49.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.727+0000 7f94c21d2700 1 -- 172.21.15.3:0/4193211516 wait complete. 2023-03-03T06:25:49.767 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-03-03T06:25:49.767 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:25:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.975+0000 7fe87f1c3700 1 Processor -- start 2023-03-03T06:25:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.979+0000 7fe87f1c3700 1 -- start start 2023-03-03T06:25:49.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87f1c3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe878159070 0x7fe87815d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87f1c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe878159440 0x7fe8780afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87f1c3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87815dc60 con 0x7fe8780ca9b0 2023-03-03T06:25:49.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87f1c3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8780ae720 con 0x7fe878159440 2023-03-03T06:25:49.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87f1c3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8780cae40 con 0x7fe878159070 2023-03-03T06:25:49.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87d9c0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe878159440 0x7fe8780afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:49.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87e9c2700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe8780ca9b0 0x7fe8780ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:56650/0 (socket says 172.21.15.3:56650) 2023-03-03T06:25:49.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87e9c2700 1 -- 172.21.15.3:0/2607446165 learned_addr learned my addr 172.21.15.3:0/2607446165 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:49.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87e1c1700 1 --2- 172.21.15.3:0/2607446165 >> v2:172.21.15.32:3300/0 conn(0x7fe878159070 0x7fe87815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:49.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87d9c0700 1 -- 172.21.15.3:0/2607446165 >> v2:172.21.15.32:3300/0 conn(0x7fe878159070 msgr2=0x7fe87815d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:49.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87d9c0700 1 --2- 172.21.15.3:0/2607446165 >> v2:172.21.15.32:3300/0 conn(0x7fe878159070 0x7fe87815d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87d9c0700 1 -- 172.21.15.3:0/2607446165 >> v1:172.21.15.32:6789/0 conn(0x7fe8780ca9b0 legacy=0x7fe8780ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:49.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87d9c0700 1 -- 172.21.15.3:0/2607446165 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8780ade80 con 0x7fe878159440 2023-03-03T06:25:49.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87d1bf700 1 -- 172.21.15.3:0/2607446165 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260213186 0 0) 0x7fe870001490 con 0x7fe8780ca9b0 2023-03-03T06:25:49.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.983+0000 7fe87d9c0700 1 --2- 172.21.15.3:0/2607446165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe878159440 0x7fe8780afb60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8780cb180 tx=0x7fe86800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4241ab5cc42aed server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:49.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.987+0000 7fe87d1bf700 1 -- 172.21.15.3:0/2607446165 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868010070 con 0x7fe878159440 2023-03-03T06:25:49.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.987+0000 7fe87d1bf700 1 -- 172.21.15.3:0/2607446165 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86800db20 con 0x7fe878159440 2023-03-03T06:25:49.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.987+0000 7fe87d1bf700 1 -- 172.21.15.3:0/2607446165 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868014430 con 0x7fe878159440 2023-03-03T06:25:49.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.987+0000 7fe87f1c3700 1 -- 172.21.15.3:0/2607446165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe878159440 msgr2=0x7fe8780afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:49.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.987+0000 7fe87f1c3700 1 --2- 172.21.15.3:0/2607446165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe878159440 0x7fe8780afb60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8780cb180 tx=0x7fe86800d330 comp rx=0 tx=0).stop 2023-03-03T06:25:49.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.987+0000 7fe87f1c3700 1 -- 172.21.15.3:0/2607446165 shutdown_connections 2023-03-03T06:25:49.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.987+0000 7fe87f1c3700 1 --2- 172.21.15.3:0/2607446165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe878159440 0x7fe8780afb60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.987+0000 7fe87f1c3700 1 --2- 172.21.15.3:0/2607446165 >> v2:172.21.15.32:3300/0 conn(0x7fe878159070 0x7fe87815d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:49.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.987+0000 7fe87f1c3700 1 -- 172.21.15.3:0/2607446165 >> 172.21.15.3:0/2607446165 conn(0x7fe878070a30 msgr2=0x7fe878070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:49.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.987+0000 7fe87f1c3700 1 -- 172.21.15.3:0/2607446165 shutdown_connections 2023-03-03T06:25:49.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.987+0000 7fe87f1c3700 1 -- 172.21.15.3:0/2607446165 wait complete. 2023-03-03T06:25:49.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.991+0000 7fe87f1c3700 1 Processor -- start 2023-03-03T06:25:49.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.991+0000 7fe87f1c3700 1 -- start start 2023-03-03T06:25:49.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87f1c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8780ca9b0 0x7fe8780b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87f1c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe878159070 0x7fe878154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87f1c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe878159440 0x7fe8781576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:49.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87f1c3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8781f1d60 con 0x7fe878159070 2023-03-03T06:25:49.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87f1c3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8781f2040 con 0x7fe8780ca9b0 2023-03-03T06:25:50.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87f1c3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8781f2320 con 0x7fe878159440 2023-03-03T06:25:50.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87d9c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe878159070 0x7fe878154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87d9c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe878159070 0x7fe878154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43782/0 (socket says 172.21.15.3:43782) 2023-03-03T06:25:50.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87d9c0700 1 -- 172.21.15.3:0/214357799 learned_addr learned my addr 172.21.15.3:0/214357799 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:50.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87e1c1700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8780ca9b0 0x7fe8780b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87e9c2700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe878159440 0x7fe8781576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87d9c0700 1 -- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe878159440 msgr2=0x7fe8781576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87d9c0700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe878159440 0x7fe8781576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87d9c0700 1 -- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8780ca9b0 msgr2=0x7fe8780b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87d9c0700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8780ca9b0 0x7fe8780b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87d9c0700 1 -- 172.21.15.3:0/214357799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86800ba70 con 0x7fe878159070 2023-03-03T06:25:50.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87e1c1700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8780ca9b0 0x7fe8780b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:50.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87d9c0700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe878159070 0x7fe878154850 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe87806aba0 tx=0x7fe868004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:50.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe86effd700 1 -- 172.21.15.3:0/214357799 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868014430 con 0x7fe878159070 2023-03-03T06:25:50.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe86effd700 1 -- 172.21.15.3:0/214357799 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe868003ad0 con 0x7fe878159070 2023-03-03T06:25:50.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe87f1c3700 1 -- 172.21.15.3:0/214357799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8780ae440 con 0x7fe878159070 2023-03-03T06:25:50.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.995+0000 7fe86effd700 1 -- 172.21.15.3:0/214357799 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868003e40 con 0x7fe878159070 2023-03-03T06:25:50.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.999+0000 7fe87f1c3700 1 -- 172.21.15.3:0/214357799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8780ae160 con 0x7fe878159070 2023-03-03T06:25:50.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:49.999+0000 7fe87f1c3700 1 -- 172.21.15.3:0/214357799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8780adbd0 con 0x7fe878159070 2023-03-03T06:25:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.003+0000 7fe86effd700 1 -- 172.21.15.3:0/214357799 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe8680145e0 con 0x7fe878159070 2023-03-03T06:25:50.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.003+0000 7fe86effd700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe85c064f30 0x7fe85c0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:50.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.003+0000 7fe86effd700 1 -- 172.21.15.3:0/214357799 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe868083cf0 con 0x7fe878159070 2023-03-03T06:25:50.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.003+0000 7fe87e1c1700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe85c064f30 0x7fe85c0673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.007+0000 7fe87e1c1700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe85c064f30 0x7fe85c0673e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe874009c40 tx=0x7fe874008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:50.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.011+0000 7fe86effd700 1 -- 172.21.15.3:0/214357799 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe868051b30 con 0x7fe878159070 2023-03-03T06:25:50.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.191+0000 7fe87f1c3700 1 -- 172.21.15.3:0/214357799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe878073e90 con 0x7fe878159070 2023-03-03T06:25:50.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.195+0000 7fe86effd700 1 -- 172.21.15.3:0/214357799 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19556 (secure 0 0 0) 0x7fe868055150 con 0x7fe878159070 2023-03-03T06:25:50.199 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:50.199 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:18.863350+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:25:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.199+0000 7fe87f1c3700 1 -- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe85c064f30 msgr2=0x7fe85c0673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.199+0000 7fe87f1c3700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe85c064f30 0x7fe85c0673e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe874009c40 tx=0x7fe874008040 comp rx=0 tx=0).stop 2023-03-03T06:25:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.199+0000 7fe87f1c3700 1 -- 172.21.15.3:0/214357799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe878159070 msgr2=0x7fe878154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.199+0000 7fe87f1c3700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe878159070 0x7fe878154850 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe87806aba0 tx=0x7fe868004770 comp rx=0 tx=0).stop 2023-03-03T06:25:50.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.199+0000 7fe87f1c3700 1 -- 172.21.15.3:0/214357799 shutdown_connections 2023-03-03T06:25:50.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.199+0000 7fe87f1c3700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe85c064f30 0x7fe85c0673e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.199+0000 7fe87f1c3700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe878159440 0x7fe8781576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.199+0000 7fe87f1c3700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe878159070 0x7fe878154850 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.199+0000 7fe87f1c3700 1 --2- 172.21.15.3:0/214357799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8780ca9b0 0x7fe8780b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.199+0000 7fe87f1c3700 1 -- 172.21.15.3:0/214357799 >> 172.21.15.3:0/214357799 conn(0x7fe878070a30 msgr2=0x7fe87806e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:50.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.199+0000 7fe87f1c3700 1 -- 172.21.15.3:0/214357799 shutdown_connections 2023-03-03T06:25:50.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.199+0000 7fe87f1c3700 1 -- 172.21.15.3:0/214357799 wait complete. 2023-03-03T06:25:50.236 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-03T06:25:11.468534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-03T06:25:12.813283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-03T06:25:16.382800+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-03T06:25:50.237 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:25:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.455+0000 7f5a4cf69700 1 Processor -- start 2023-03-03T06:25:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.455+0000 7f5a4cf69700 1 -- start start 2023-03-03T06:25:50.463 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.459+0000 7f5a4cf69700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5a4815d6e0 0x7f5a480cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:50.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.459+0000 7f5a4cf69700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a480cfd80 0x7f5a480d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:50.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.459+0000 7f5a4cf69700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a480cf3b0 con 0x7f5a48156ba0 2023-03-03T06:25:50.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.459+0000 7f5a4cf69700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a48156f70 con 0x7f5a4815d6e0 2023-03-03T06:25:50.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.459+0000 7f5a4cf69700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480cd320 con 0x7f5a480cfd80 2023-03-03T06:25:50.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.459+0000 7f5a47fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a480cfd80 0x7f5a480d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.459+0000 7f5a46ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5a4815d6e0 0x7f5a480cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.459+0000 7f5a47fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a480cfd80 0x7f5a480d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:56360/0 (socket says 172.21.15.3:56360) 2023-03-03T06:25:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.459+0000 7f5a47fff700 1 -- 172.21.15.3:0/1992574154 learned_addr learned my addr 172.21.15.3:0/1992574154 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.459+0000 7f5a477fe700 1 --1- 172.21.15.3:0/1992574154 >> v1:172.21.15.32:6789/0 conn(0x7f5a48156ba0 0x7f5a48071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:36134/0 (socket says 172.21.15.3:36134) 2023-03-03T06:25:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a47fff700 1 -- 172.21.15.3:0/1992574154 >> v2:172.21.15.32:3300/0 conn(0x7f5a4815d6e0 msgr2=0x7f5a480cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:50.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a47fff700 1 --2- 172.21.15.3:0/1992574154 >> v2:172.21.15.32:3300/0 conn(0x7f5a4815d6e0 0x7f5a480cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a47fff700 1 -- 172.21.15.3:0/1992574154 >> v1:172.21.15.32:6789/0 conn(0x7f5a48156ba0 legacy=0x7f5a48071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a47fff700 1 -- 172.21.15.3:0/1992574154 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a480cd600 con 0x7f5a480cfd80 2023-03-03T06:25:50.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a46ffd700 1 --2- 172.21.15.3:0/1992574154 >> v2:172.21.15.32:3300/0 conn(0x7f5a4815d6e0 0x7f5a480cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:50.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a47fff700 1 --2- 172.21.15.3:0/1992574154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a480cfd80 0x7f5a480d0b30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009e50 tx=0x7f5a4000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10eee052a1d7625e server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:50.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a467fc700 1 -- 172.21.15.3:0/1992574154 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000e070 con 0x7f5a480cfd80 2023-03-03T06:25:50.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a467fc700 1 -- 172.21.15.3:0/1992574154 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4000b910 con 0x7f5a480cfd80 2023-03-03T06:25:50.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a467fc700 1 -- 172.21.15.3:0/1992574154 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000bcc0 con 0x7f5a480cfd80 2023-03-03T06:25:50.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a4cf69700 1 -- 172.21.15.3:0/1992574154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a480cfd80 msgr2=0x7f5a480d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a4cf69700 1 --2- 172.21.15.3:0/1992574154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a480cfd80 0x7f5a480d0b30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009e50 tx=0x7f5a4000b330 comp rx=0 tx=0).stop 2023-03-03T06:25:50.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a4cf69700 1 -- 172.21.15.3:0/1992574154 shutdown_connections 2023-03-03T06:25:50.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a4cf69700 1 --2- 172.21.15.3:0/1992574154 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a480cfd80 0x7f5a480d0b30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a4cf69700 1 --2- 172.21.15.3:0/1992574154 >> v2:172.21.15.32:3300/0 conn(0x7f5a4815d6e0 0x7f5a480cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a4cf69700 1 -- 172.21.15.3:0/1992574154 >> 172.21.15.3:0/1992574154 conn(0x7f5a4806e5c0 msgr2=0x7f5a4806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:50.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a4cf69700 1 -- 172.21.15.3:0/1992574154 shutdown_connections 2023-03-03T06:25:50.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.463+0000 7f5a4cf69700 1 -- 172.21.15.3:0/1992574154 wait complete. 2023-03-03T06:25:50.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.467+0000 7f5a4cf69700 1 Processor -- start 2023-03-03T06:25:50.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.467+0000 7f5a4cf69700 1 -- start start 2023-03-03T06:25:50.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.467+0000 7f5a4cf69700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a480cfd80 0x7f5a48150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:50.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.467+0000 7f5a4cf69700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a48156ba0 0x7f5a480cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:50.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.467+0000 7f5a4cf69700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a4815d6e0 0x7f5a480cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:50.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.467+0000 7f5a4cf69700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a481f63f0 con 0x7f5a480cfd80 2023-03-03T06:25:50.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.467+0000 7f5a4cf69700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480b4cb0 con 0x7f5a4815d6e0 2023-03-03T06:25:50.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.467+0000 7f5a4cf69700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480b4f90 con 0x7f5a48156ba0 2023-03-03T06:25:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.467+0000 7f5a477fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a480cfd80 0x7f5a48150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.467+0000 7f5a477fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a480cfd80 0x7f5a48150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47878/0 (socket says 172.21.15.3:47878) 2023-03-03T06:25:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.467+0000 7f5a477fe700 1 -- 172.21.15.3:0/2404011556 learned_addr learned my addr 172.21.15.3:0/2404011556 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:50.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a46ffd700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a48156ba0 0x7f5a480cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a47fff700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a4815d6e0 0x7f5a480cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a477fe700 1 -- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a48156ba0 msgr2=0x7f5a480cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a477fe700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a48156ba0 0x7f5a480cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a477fe700 1 -- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a4815d6e0 msgr2=0x7f5a480cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a477fe700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a4815d6e0 0x7f5a480cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a477fe700 1 -- 172.21.15.3:0/2404011556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c0094b0 con 0x7f5a480cfd80 2023-03-03T06:25:50.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a47fff700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a4815d6e0 0x7f5a480cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:50.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a477fe700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a480cfd80 0x7f5a48150280 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c500 tx=0x7f5a3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:50.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a2ffff700 1 -- 172.21.15.3:0/2404011556 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38005cd0 con 0x7f5a480cfd80 2023-03-03T06:25:50.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a4cf69700 1 -- 172.21.15.3:0/2404011556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a40009b30 con 0x7f5a480cfd80 2023-03-03T06:25:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a2ffff700 1 -- 172.21.15.3:0/2404011556 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a38005e30 con 0x7f5a480cfd80 2023-03-03T06:25:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a2ffff700 1 -- 172.21.15.3:0/2404011556 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380072f0 con 0x7f5a480cfd80 2023-03-03T06:25:50.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.471+0000 7f5a4cf69700 1 -- 172.21.15.3:0/2404011556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a480cef40 con 0x7f5a480cfd80 2023-03-03T06:25:50.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.475+0000 7f5a4cf69700 1 -- 172.21.15.3:0/2404011556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a480cd8e0 con 0x7f5a480cfd80 2023-03-03T06:25:50.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.475+0000 7f5a2ffff700 1 -- 172.21.15.3:0/2404011556 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5a38019030 con 0x7f5a480cfd80 2023-03-03T06:25:50.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.479+0000 7f5a2ffff700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5a34064f10 0x7f5a340673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:50.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.479+0000 7f5a2ffff700 1 -- 172.21.15.3:0/2404011556 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5a38081670 con 0x7f5a480cfd80 2023-03-03T06:25:50.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.479+0000 7f5a46ffd700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5a34064f10 0x7f5a340673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.479+0000 7f5a46ffd700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5a34064f10 0x7f5a340673c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5a480d2370 tx=0x7f5a3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:50.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.483+0000 7f5a2ffff700 1 -- 172.21.15.3:0/2404011556 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5a3804f430 con 0x7f5a480cfd80 2023-03-03T06:25:50.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.659+0000 7f5a4cf69700 1 -- 172.21.15.3:0/2404011556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a480cf0d0 con 0x7f5a480cfd80 2023-03-03T06:25:50.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.659+0000 7f5a2ffff700 1 -- 172.21.15.3:0/2404011556 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a380170a0 con 0x7f5a480cfd80 2023-03-03T06:25:50.664 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:25:50.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.667+0000 7f5a4cf69700 1 -- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5a34064f10 msgr2=0x7f5a340673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.667+0000 7f5a4cf69700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5a34064f10 0x7f5a340673c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5a480d2370 tx=0x7f5a3c008040 comp rx=0 tx=0).stop 2023-03-03T06:25:50.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.667+0000 7f5a4cf69700 1 -- 172.21.15.3:0/2404011556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a480cfd80 msgr2=0x7f5a48150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.667+0000 7f5a4cf69700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a480cfd80 0x7f5a48150280 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c500 tx=0x7f5a3800c8c0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.667+0000 7f5a4cf69700 1 -- 172.21.15.3:0/2404011556 shutdown_connections 2023-03-03T06:25:50.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.667+0000 7f5a4cf69700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5a34064f10 0x7f5a340673c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.667+0000 7f5a4cf69700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5a4815d6e0 0x7f5a480cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.667+0000 7f5a4cf69700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5a48156ba0 0x7f5a480cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.667+0000 7f5a4cf69700 1 --2- 172.21.15.3:0/2404011556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a480cfd80 0x7f5a48150280 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.667+0000 7f5a4cf69700 1 -- 172.21.15.3:0/2404011556 >> 172.21.15.3:0/2404011556 conn(0x7f5a4806e5c0 msgr2=0x7f5a48151140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:50.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.667+0000 7f5a4cf69700 1 -- 172.21.15.3:0/2404011556 shutdown_connections 2023-03-03T06:25:50.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.667+0000 7f5a4cf69700 1 -- 172.21.15.3:0/2404011556 wait complete. 2023-03-03T06:25:50.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:25:50.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.923+0000 7f28c1561700 1 Processor -- start 2023-03-03T06:25:50.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.923+0000 7f28c1561700 1 -- start start 2023-03-03T06:25:50.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28c1561700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28bc0cced0 0x7f28bc0cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:50.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28c1561700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f28bc0cc980 0x7f28bc0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:50.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28c1561700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28bc0b0330 con 0x7f28bc1590b0 2023-03-03T06:25:50.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28c1561700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28bc15dd00 con 0x7f28bc0cc980 2023-03-03T06:25:50.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28c1561700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0b4160 con 0x7f28bc0cced0 2023-03-03T06:25:50.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28bbfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f28bc0cc980 0x7f28bc0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28ba7fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28bc0cced0 0x7f28bc0cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28baffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f28bc1590b0 0x7f28bc0ce680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:36150/0 (socket says 172.21.15.3:36150) 2023-03-03T06:25:50.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28baffd700 1 -- 172.21.15.3:0/2012914808 learned_addr learned my addr 172.21.15.3:0/2012914808 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:50.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28bbfff700 1 -- 172.21.15.3:0/2012914808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28bc0cced0 msgr2=0x7f28bc0cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:50.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28bbfff700 1 --2- 172.21.15.3:0/2012914808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28bc0cced0 0x7f28bc0cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28bbfff700 1 -- 172.21.15.3:0/2012914808 >> v1:172.21.15.32:6789/0 conn(0x7f28bc1590b0 legacy=0x7f28bc0ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28bbfff700 1 -- 172.21.15.3:0/2012914808 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc15da20 con 0x7f28bc0cc980 2023-03-03T06:25:50.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.927+0000 7f28bbfff700 1 --2- 172.21.15.3:0/2012914808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28bc0cc980 0x7f28bc0cc2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009d90 tx=0x7f28b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2035cb88da1f8e server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.931+0000 7f28b9ffb700 1 -- 172.21.15.3:0/2012914808 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b400e070 con 0x7f28bc0cc980 2023-03-03T06:25:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.931+0000 7f28b9ffb700 1 -- 172.21.15.3:0/2012914808 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b400bb20 con 0x7f28bc0cc980 2023-03-03T06:25:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.931+0000 7f28b9ffb700 1 -- 172.21.15.3:0/2012914808 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4012430 con 0x7f28bc0cc980 2023-03-03T06:25:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.931+0000 7f28c1561700 1 -- 172.21.15.3:0/2012914808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28bc0cc980 msgr2=0x7f28bc0cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.931+0000 7f28c1561700 1 --2- 172.21.15.3:0/2012914808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28bc0cc980 0x7f28bc0cc2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009d90 tx=0x7f28b400b330 comp rx=0 tx=0).stop 2023-03-03T06:25:50.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.931+0000 7f28c1561700 1 -- 172.21.15.3:0/2012914808 shutdown_connections 2023-03-03T06:25:50.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.931+0000 7f28c1561700 1 --2- 172.21.15.3:0/2012914808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28bc0cc980 0x7f28bc0cc2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.931+0000 7f28c1561700 1 --2- 172.21.15.3:0/2012914808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28bc0cced0 0x7f28bc0cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.931+0000 7f28c1561700 1 -- 172.21.15.3:0/2012914808 >> 172.21.15.3:0/2012914808 conn(0x7f28bc06b0d0 msgr2=0x7f28bc0cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:50.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.931+0000 7f28c1561700 1 -- 172.21.15.3:0/2012914808 shutdown_connections 2023-03-03T06:25:50.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.931+0000 7f28c1561700 1 -- 172.21.15.3:0/2012914808 wait complete. 2023-03-03T06:25:50.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.935+0000 7f28c1561700 1 Processor -- start 2023-03-03T06:25:50.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.935+0000 7f28c1561700 1 -- start start 2023-03-03T06:25:50.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28c1561700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28bc0cc980 0x7f28bc157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:50.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28c1561700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28bc0cced0 0x7f28bc0ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:50.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28c1561700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28bc1590b0 0x7f28bc0b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:50.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28c1561700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0c49d0 con 0x7f28bc1590b0 2023-03-03T06:25:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28c1561700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0ad410 con 0x7f28bc0cc980 2023-03-03T06:25:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28c1561700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc157a40 con 0x7f28bc0cced0 2023-03-03T06:25:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28bc1590b0 0x7f28bc0b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28bc1590b0 0x7f28bc0b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47894/0 (socket says 172.21.15.3:47894) 2023-03-03T06:25:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28bbfff700 1 -- 172.21.15.3:0/1060728558 learned_addr learned my addr 172.21.15.3:0/1060728558 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:50.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28baffd700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28bc0cc980 0x7f28bc157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28ba7fc700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28bc0cced0 0x7f28bc0ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28bbfff700 1 -- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28bc0cced0 msgr2=0x7f28bc0ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28bbfff700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28bc0cced0 0x7f28bc0ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28bbfff700 1 -- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28bc0cc980 msgr2=0x7f28bc157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28bbfff700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28bc0cc980 0x7f28bc157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28bbfff700 1 -- 172.21.15.3:0/1060728558 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b4009a70 con 0x7f28bc1590b0 2023-03-03T06:25:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28ba7fc700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28bc0cced0 0x7f28bc0ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28bbfff700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28bc1590b0 0x7f28bc0b1b90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0ce630 tx=0x7f28b4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:50.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28b37fe700 1 -- 172.21.15.3:0/1060728558 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4012430 con 0x7f28bc1590b0 2023-03-03T06:25:50.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28b37fe700 1 -- 172.21.15.3:0/1060728558 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b4020070 con 0x7f28bc1590b0 2023-03-03T06:25:50.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28c1561700 1 -- 172.21.15.3:0/1060728558 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28bc0b3d80 con 0x7f28bc1590b0 2023-03-03T06:25:50.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28b37fe700 1 -- 172.21.15.3:0/1060728558 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4004280 con 0x7f28bc1590b0 2023-03-03T06:25:50.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.939+0000 7f28c1561700 1 -- 172.21.15.3:0/1060728558 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28bc0b3aa0 con 0x7f28bc1590b0 2023-03-03T06:25:50.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.943+0000 7f28c1561700 1 -- 172.21.15.3:0/1060728558 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28bc0ced90 con 0x7f28bc1590b0 2023-03-03T06:25:50.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.943+0000 7f28b37fe700 1 -- 172.21.15.3:0/1060728558 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f28b402e070 con 0x7f28bc1590b0 2023-03-03T06:25:50.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.947+0000 7f28b37fe700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f28a4064ee0 0x7f28a4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:50.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.947+0000 7f28b37fe700 1 -- 172.21.15.3:0/1060728558 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f28b4022070 con 0x7f28bc1590b0 2023-03-03T06:25:50.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.947+0000 7f28baffd700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f28a4064ee0 0x7f28a4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:50.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.947+0000 7f28baffd700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f28a4064ee0 0x7f28a4067390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005930 tx=0x7f28ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:50.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:50.951+0000 7f28b37fe700 1 -- 172.21.15.3:0/1060728558 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f28b4051d80 con 0x7f28bc1590b0 2023-03-03T06:25:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.131+0000 7f28c1561700 1 -- 172.21.15.3:0/1060728558 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f28bc04e180 con 0x7f28bc1590b0 2023-03-03T06:25:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.131+0000 7f28b37fe700 1 -- 172.21.15.3:0/1060728558 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f28b402c020 con 0x7f28bc1590b0 2023-03-03T06:25:51.136 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:25:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.135+0000 7f28c1561700 1 -- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f28a4064ee0 msgr2=0x7f28a4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.135+0000 7f28c1561700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f28a4064ee0 0x7f28a4067390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005930 tx=0x7f28ac009040 comp rx=0 tx=0).stop 2023-03-03T06:25:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.135+0000 7f28c1561700 1 -- 172.21.15.3:0/1060728558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28bc1590b0 msgr2=0x7f28bc0b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.135+0000 7f28c1561700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28bc1590b0 0x7f28bc0b1b90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0ce630 tx=0x7f28b4004af0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.135+0000 7f28c1561700 1 -- 172.21.15.3:0/1060728558 shutdown_connections 2023-03-03T06:25:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.135+0000 7f28c1561700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f28a4064ee0 0x7f28a4067390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.135+0000 7f28c1561700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28bc1590b0 0x7f28bc0b1b90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.135+0000 7f28c1561700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f28bc0cced0 0x7f28bc0ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.135+0000 7f28c1561700 1 --2- 172.21.15.3:0/1060728558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28bc0cc980 0x7f28bc157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.135+0000 7f28c1561700 1 -- 172.21.15.3:0/1060728558 >> 172.21.15.3:0/1060728558 conn(0x7f28bc06b0d0 msgr2=0x7f28bc0ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.135+0000 7f28c1561700 1 -- 172.21.15.3:0/1060728558 shutdown_connections 2023-03-03T06:25:51.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.135+0000 7f28c1561700 1 -- 172.21.15.3:0/1060728558 wait complete. 2023-03-03T06:25:51.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-03T06:25:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.375+0000 7f5f3e2ff700 1 Processor -- start 2023-03-03T06:25:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.375+0000 7f5f3e2ff700 1 -- start start 2023-03-03T06:25:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f3e2ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f3806a780 0x7f5f380cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f3e2ff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f380cb9a0 0x7f5f380709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f3e2ff700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f380cf5f0 con 0x7f5f38070ef0 2023-03-03T06:25:51.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f3e2ff700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f380caac0 con 0x7f5f380cb9a0 2023-03-03T06:25:51.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f3e2ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f38071370 con 0x7f5f3806a780 2023-03-03T06:25:51.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f3dafe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5f38070ef0 0x7f5f3806c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46918/0 (socket says 172.21.15.3:46918) 2023-03-03T06:25:51.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f3dafe700 1 -- 172.21.15.3:0/400380253 learned_addr learned my addr 172.21.15.3:0/400380253 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:51.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f3cafc700 1 --2- 172.21.15.3:0/400380253 >> v2:172.21.15.32:3300/0 conn(0x7f5f380cb9a0 0x7f5f380709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:51.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f3d2fd700 1 --2- 172.21.15.3:0/400380253 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f3806a780 0x7f5f380cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f2ffff700 1 -- 172.21.15.3:0/400380253 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924616431 0 0) 0x7f5f380cf5f0 con 0x7f5f38070ef0 2023-03-03T06:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f2ffff700 1 -- 172.21.15.3:0/400380253 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f240035d0 con 0x7f5f38070ef0 2023-03-03T06:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f3d2fd700 1 -- 172.21.15.3:0/400380253 >> v2:172.21.15.32:3300/0 conn(0x7f5f380cb9a0 msgr2=0x7f5f380709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f3d2fd700 1 --2- 172.21.15.3:0/400380253 >> v2:172.21.15.32:3300/0 conn(0x7f5f380cb9a0 0x7f5f380709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f3d2fd700 1 -- 172.21.15.3:0/400380253 >> v1:172.21.15.3:6789/0 conn(0x7f5f38070ef0 legacy=0x7f5f3806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.379+0000 7f5f3d2fd700 1 -- 172.21.15.3:0/400380253 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3815c230 con 0x7f5f3806a780 2023-03-03T06:25:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.383+0000 7f5f3d2fd700 1 --2- 172.21.15.3:0/400380253 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f3806a780 0x7f5f380cace0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009d90 tx=0x7f5f3400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a63274cfdc778a1 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.383+0000 7f5f2ffff700 1 -- 172.21.15.3:0/400380253 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400e070 con 0x7f5f3806a780 2023-03-03T06:25:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.383+0000 7f5f2ffff700 1 -- 172.21.15.3:0/400380253 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3400bb20 con 0x7f5f3806a780 2023-03-03T06:25:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.383+0000 7f5f2ffff700 1 -- 172.21.15.3:0/400380253 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34012430 con 0x7f5f3806a780 2023-03-03T06:25:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.383+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/400380253 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f3806a780 msgr2=0x7f5f380cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.383+0000 7f5f3e2ff700 1 --2- 172.21.15.3:0/400380253 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f3806a780 0x7f5f380cace0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009d90 tx=0x7f5f3400b330 comp rx=0 tx=0).stop 2023-03-03T06:25:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.383+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/400380253 shutdown_connections 2023-03-03T06:25:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.383+0000 7f5f3e2ff700 1 --2- 172.21.15.3:0/400380253 >> v2:172.21.15.32:3300/0 conn(0x7f5f380cb9a0 0x7f5f380709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.383+0000 7f5f3e2ff700 1 --2- 172.21.15.3:0/400380253 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f3806a780 0x7f5f380cace0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.383+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/400380253 >> 172.21.15.3:0/400380253 conn(0x7f5f380607c0 msgr2=0x7f5f38060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.383+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/400380253 shutdown_connections 2023-03-03T06:25:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.383+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/400380253 wait complete. 2023-03-03T06:25:51.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3e2ff700 1 Processor -- start 2023-03-03T06:25:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3e2ff700 1 -- start start 2023-03-03T06:25:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3e2ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3806a780 0x7f5f38154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3e2ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f38070ef0 0x7f5f38155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3e2ff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f380cb9a0 0x7f5f3806bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:51.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3e2ff700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380c5260 con 0x7f5f3806a780 2023-03-03T06:25:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3e2ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380c49c0 con 0x7f5f38070ef0 2023-03-03T06:25:51.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3e2ff700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380c4ca0 con 0x7f5f380cb9a0 2023-03-03T06:25:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3d2fd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3806a780 0x7f5f38154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3d2fd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3806a780 0x7f5f38154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47900/0 (socket says 172.21.15.3:47900) 2023-03-03T06:25:51.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3d2fd700 1 -- 172.21.15.3:0/2819621554 learned_addr learned my addr 172.21.15.3:0/2819621554 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:51.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3dafe700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f380cb9a0 0x7f5f3806bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:51.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3cafc700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f38070ef0 0x7f5f38155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:51.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3d2fd700 1 -- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f380cb9a0 msgr2=0x7f5f3806bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:51.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3d2fd700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f380cb9a0 0x7f5f3806bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3d2fd700 1 -- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f38070ef0 msgr2=0x7f5f38155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:51.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.387+0000 7f5f3d2fd700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f38070ef0 0x7f5f38155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.391+0000 7f5f3d2fd700 1 -- 172.21.15.3:0/2819621554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f34009a70 con 0x7f5f3806a780 2023-03-03T06:25:51.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.391+0000 7f5f3dafe700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f380cb9a0 0x7f5f3806bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:51.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.391+0000 7f5f3d2fd700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3806a780 0x7f5f38154180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f34014040 tx=0x7f5f34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:51.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.391+0000 7f5f2dffb700 1 -- 172.21.15.3:0/2819621554 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34012430 con 0x7f5f3806a780 2023-03-03T06:25:51.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.391+0000 7f5f2dffb700 1 -- 172.21.15.3:0/2819621554 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f34020070 con 0x7f5f3806a780 2023-03-03T06:25:51.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.391+0000 7f5f2dffb700 1 -- 172.21.15.3:0/2819621554 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34004280 con 0x7f5f3806a780 2023-03-03T06:25:51.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.391+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/2819621554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f380cbf60 con 0x7f5f3806a780 2023-03-03T06:25:51.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.391+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/2819621554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f380cd130 con 0x7f5f3806a780 2023-03-03T06:25:51.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.395+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/2819621554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f3815c4f0 con 0x7f5f3806a780 2023-03-03T06:25:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.395+0000 7f5f2dffb700 1 -- 172.21.15.3:0/2819621554 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5f3402e070 con 0x7f5f3806a780 2023-03-03T06:25:51.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.399+0000 7f5f2dffb700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5f1c064ee0 0x7f5f1c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:51.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.399+0000 7f5f2dffb700 1 -- 172.21.15.3:0/2819621554 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5f34022070 con 0x7f5f3806a780 2023-03-03T06:25:51.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.399+0000 7f5f3cafc700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5f1c064ee0 0x7f5f1c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.399+0000 7f5f3cafc700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5f1c064ee0 0x7f5f1c067390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5f380c2250 tx=0x7f5f28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:51.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.403+0000 7f5f2dffb700 1 -- 172.21.15.3:0/2819621554 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5f34051da0 con 0x7f5f3806a780 2023-03-03T06:25:51.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.579+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/2819621554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f38073e90 con 0x7f5f3806a780 2023-03-03T06:25:51.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.579+0000 7f5f2dffb700 1 -- 172.21.15.3:0/2819621554 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f340553c0 con 0x7f5f3806a780 2023-03-03T06:25:51.583 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:25:51.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.583+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5f1c064ee0 msgr2=0x7f5f1c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:51.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.583+0000 7f5f3e2ff700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5f1c064ee0 0x7f5f1c067390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5f380c2250 tx=0x7f5f28008040 comp rx=0 tx=0).stop 2023-03-03T06:25:51.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.583+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/2819621554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3806a780 msgr2=0x7f5f38154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:51.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.583+0000 7f5f3e2ff700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3806a780 0x7f5f38154180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f34014040 tx=0x7f5f34004af0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.583+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/2819621554 shutdown_connections 2023-03-03T06:25:51.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.583+0000 7f5f3e2ff700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5f1c064ee0 0x7f5f1c067390 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.583+0000 7f5f3e2ff700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f380cb9a0 0x7f5f3806bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.583+0000 7f5f3e2ff700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f38070ef0 0x7f5f38155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.583+0000 7f5f3e2ff700 1 --2- 172.21.15.3:0/2819621554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3806a780 0x7f5f38154180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.583+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/2819621554 >> 172.21.15.3:0/2819621554 conn(0x7f5f380607c0 msgr2=0x7f5f380c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:51.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.583+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/2819621554 shutdown_connections 2023-03-03T06:25:51.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.583+0000 7f5f3e2ff700 1 -- 172.21.15.3:0/2819621554 wait complete. 2023-03-03T06:25:51.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-03T06:25:51.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.827+0000 7fd5ae934700 1 Processor -- start 2023-03-03T06:25:51.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.827+0000 7fd5ae934700 1 -- start start 2023-03-03T06:25:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ae934700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd5a8156ba0 0x7fd5a8071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ae934700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5a80d0b30 0x7fd5a80d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:51.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ae934700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a80cf3b0 con 0x7fd5a815d6e0 2023-03-03T06:25:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ae934700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a80d14f0 con 0x7fd5a8156ba0 2023-03-03T06:25:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ae934700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a8156f70 con 0x7fd5a80d0b30 2023-03-03T06:25:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ad131700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd5a815d6e0 0x7fd5a80cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46928/0 (socket says 172.21.15.3:46928) 2023-03-03T06:25:51.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ad131700 1 -- 172.21.15.3:0/2431987401 learned_addr learned my addr 172.21.15.3:0/2431987401 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ad932700 1 --2- 172.21.15.3:0/2431987401 >> v2:172.21.15.3:3300/0 conn(0x7fd5a8156ba0 0x7fd5a8071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ae133700 1 --2- 172.21.15.3:0/2431987401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5a80d0b30 0x7fd5a80d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:51.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ac930700 1 -- 172.21.15.3:0/2431987401 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845881372 0 0) 0x7fd5a80cf3b0 con 0x7fd5a815d6e0 2023-03-03T06:25:51.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ac930700 1 -- 172.21.15.3:0/2431987401 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5940035d0 con 0x7fd5a815d6e0 2023-03-03T06:25:51.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ae133700 1 -- 172.21.15.3:0/2431987401 >> v2:172.21.15.3:3300/0 conn(0x7fd5a8156ba0 msgr2=0x7fd5a8071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:51.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ae133700 1 --2- 172.21.15.3:0/2431987401 >> v2:172.21.15.3:3300/0 conn(0x7fd5a8156ba0 0x7fd5a8071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.831+0000 7fd5ae133700 1 -- 172.21.15.3:0/2431987401 >> v1:172.21.15.3:6789/0 conn(0x7fd5a815d6e0 legacy=0x7fd5a80cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:51.838 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.835+0000 7fd5ae133700 1 -- 172.21.15.3:0/2431987401 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a80cd320 con 0x7fd5a80d0b30 2023-03-03T06:25:51.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.835+0000 7fd5ae133700 1 --2- 172.21.15.3:0/2431987401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5a80d0b30 0x7fd5a80d0fb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009d90 tx=0x7fd5a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a28895eae04637b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:51.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.835+0000 7fd5ac930700 1 -- 172.21.15.3:0/2431987401 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a000e070 con 0x7fd5a80d0b30 2023-03-03T06:25:51.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.835+0000 7fd5ac930700 1 -- 172.21.15.3:0/2431987401 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a000bb20 con 0x7fd5a80d0b30 2023-03-03T06:25:51.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.835+0000 7fd5ac930700 1 -- 172.21.15.3:0/2431987401 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0012430 con 0x7fd5a80d0b30 2023-03-03T06:25:51.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.835+0000 7fd5ae934700 1 -- 172.21.15.3:0/2431987401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5a80d0b30 msgr2=0x7fd5a80d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:51.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.835+0000 7fd5ae934700 1 --2- 172.21.15.3:0/2431987401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5a80d0b30 0x7fd5a80d0fb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009d90 tx=0x7fd5a000b330 comp rx=0 tx=0).stop 2023-03-03T06:25:51.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.835+0000 7fd5ae934700 1 -- 172.21.15.3:0/2431987401 shutdown_connections 2023-03-03T06:25:51.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.835+0000 7fd5ae934700 1 --2- 172.21.15.3:0/2431987401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5a80d0b30 0x7fd5a80d0fb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.835+0000 7fd5ae934700 1 --2- 172.21.15.3:0/2431987401 >> v2:172.21.15.3:3300/0 conn(0x7fd5a8156ba0 0x7fd5a8071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.835+0000 7fd5ae934700 1 -- 172.21.15.3:0/2431987401 >> 172.21.15.3:0/2431987401 conn(0x7fd5a806e5c0 msgr2=0x7fd5a806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:51.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.839+0000 7fd5ae934700 1 -- 172.21.15.3:0/2431987401 shutdown_connections 2023-03-03T06:25:51.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.839+0000 7fd5ae934700 1 -- 172.21.15.3:0/2431987401 wait complete. 2023-03-03T06:25:51.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ae934700 1 Processor -- start 2023-03-03T06:25:51.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ae934700 1 -- start start 2023-03-03T06:25:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ae934700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d0b30 0x7fd5a80b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:51.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ae934700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5a8156ba0 0x7fd5a815e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ae934700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5a815d6e0 0x7fd5a80d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ae934700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81f5aa0 con 0x7fd5a80d0b30 2023-03-03T06:25:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ae934700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81f4e90 con 0x7fd5a815d6e0 2023-03-03T06:25:51.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ae934700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81f5170 con 0x7fd5a8156ba0 2023-03-03T06:25:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ad932700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d0b30 0x7fd5a80b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ad932700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d0b30 0x7fd5a80b82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47910/0 (socket says 172.21.15.3:47910) 2023-03-03T06:25:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ad932700 1 -- 172.21.15.3:0/896851921 learned_addr learned my addr 172.21.15.3:0/896851921 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ad131700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5a8156ba0 0x7fd5a815e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:51.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ad932700 1 -- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5a8156ba0 msgr2=0x7fd5a815e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:51.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ad932700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5a8156ba0 0x7fd5a815e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ad932700 1 -- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5a815d6e0 msgr2=0x7fd5a80d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:51.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ad932700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5a815d6e0 0x7fd5a80d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:51.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ad932700 1 -- 172.21.15.3:0/896851921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a0009a70 con 0x7fd5a80d0b30 2023-03-03T06:25:51.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ad131700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5a8156ba0 0x7fd5a815e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:25:51.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.843+0000 7fd5ad932700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d0b30 0x7fd5a80b82b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400e840 tx=0x7fd5a400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.847+0000 7fd59e7fc700 1 -- 172.21.15.3:0/896851921 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a40095a0 con 0x7fd5a80d0b30 2023-03-03T06:25:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.847+0000 7fd59e7fc700 1 -- 172.21.15.3:0/896851921 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a400c690 con 0x7fd5a80d0b30 2023-03-03T06:25:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.847+0000 7fd59e7fc700 1 -- 172.21.15.3:0/896851921 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a400ca00 con 0x7fd5a80d0b30 2023-03-03T06:25:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.847+0000 7fd5ae934700 1 -- 172.21.15.3:0/896851921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a80cd8e0 con 0x7fd5a80d0b30 2023-03-03T06:25:51.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.847+0000 7fd5ae934700 1 -- 172.21.15.3:0/896851921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a80cd600 con 0x7fd5a80d0b30 2023-03-03T06:25:51.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.847+0000 7fd5ae934700 1 -- 172.21.15.3:0/896851921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a80717c0 con 0x7fd5a80d0b30 2023-03-03T06:25:51.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.847+0000 7fd59e7fc700 1 -- 172.21.15.3:0/896851921 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd5a4019030 con 0x7fd5a80d0b30 2023-03-03T06:25:51.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.851+0000 7fd59e7fc700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd58c064f10 0x7fd58c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:51.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.851+0000 7fd59e7fc700 1 -- 172.21.15.3:0/896851921 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd5a40820c0 con 0x7fd5a80d0b30 2023-03-03T06:25:51.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.851+0000 7fd5ad131700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd58c064f10 0x7fd58c0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:51.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.855+0000 7fd5ad131700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd58c064f10 0x7fd58c0673c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd5a815a5b0 tx=0x7fd598009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:51.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:51.855+0000 7fd59e7fc700 1 -- 172.21.15.3:0/896851921 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd5a404fe80 con 0x7fd5a80d0b30 2023-03-03T06:25:52.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.035+0000 7fd5ae934700 1 -- 172.21.15.3:0/896851921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5a8073e90 con 0x7fd5a80d0b30 2023-03-03T06:25:52.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.035+0000 7fd59e7fc700 1 -- 172.21.15.3:0/896851921 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5a40534a0 con 0x7fd5a80d0b30 2023-03-03T06:25:52.042 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-03-03T06:25:52.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.039+0000 7fd5ae934700 1 -- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd58c064f10 msgr2=0x7fd58c0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:52.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.039+0000 7fd5ae934700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd58c064f10 0x7fd58c0673c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd5a815a5b0 tx=0x7fd598009040 comp rx=0 tx=0).stop 2023-03-03T06:25:52.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.039+0000 7fd5ae934700 1 -- 172.21.15.3:0/896851921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d0b30 msgr2=0x7fd5a80b82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:52.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.039+0000 7fd5ae934700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d0b30 0x7fd5a80b82b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400e840 tx=0x7fd5a400ec00 comp rx=0 tx=0).stop 2023-03-03T06:25:52.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.039+0000 7fd5ae934700 1 -- 172.21.15.3:0/896851921 shutdown_connections 2023-03-03T06:25:52.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.039+0000 7fd5ae934700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd58c064f10 0x7fd58c0673c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.039+0000 7fd5ae934700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5a815d6e0 0x7fd5a80d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.039+0000 7fd5ae934700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd5a8156ba0 0x7fd5a815e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.039+0000 7fd5ae934700 1 --2- 172.21.15.3:0/896851921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5a80d0b30 0x7fd5a80b82b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.039+0000 7fd5ae934700 1 -- 172.21.15.3:0/896851921 >> 172.21.15.3:0/896851921 conn(0x7fd5a806e5c0 msgr2=0x7fd5a8150500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.039+0000 7fd5ae934700 1 -- 172.21.15.3:0/896851921 shutdown_connections 2023-03-03T06:25:52.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.039+0000 7fd5ae934700 1 -- 172.21.15.3:0/896851921 wait complete. 2023-03-03T06:25:52.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-03T06:25:52.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.295+0000 7fe157084700 1 Processor -- start 2023-03-03T06:25:52.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.295+0000 7fe157084700 1 -- start start 2023-03-03T06:25:52.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe157084700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe150156ba0 0x7fe150071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:52.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe157084700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe15015d6e0 0x7fe1500cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:52.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe157084700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe1500cfd80 0x7fe1500d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:52.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe157084700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1500cf3b0 con 0x7fe1500cfd80 2023-03-03T06:25:52.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe157084700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe150156f70 con 0x7fe15015d6e0 2023-03-03T06:25:52.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe157084700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500cd320 con 0x7fe150156ba0 2023-03-03T06:25:52.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe156883700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe1500cfd80 0x7fe1500d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:52.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe156883700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe1500cfd80 0x7fe1500d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47922/0 (socket says 172.21.15.3:47922) 2023-03-03T06:25:52.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe156883700 1 -- 172.21.15.3:0/2182069948 learned_addr learned my addr 172.21.15.3:0/2182069948 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:52.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe156082700 1 --2- 172.21.15.3:0/2182069948 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe150156ba0 0x7fe150071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:52.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe155881700 1 --2- 172.21.15.3:0/2182069948 >> v2:172.21.15.32:3300/0 conn(0x7fe15015d6e0 0x7fe1500cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:52.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe156082700 1 -- 172.21.15.3:0/2182069948 >> v2:172.21.15.32:3300/0 conn(0x7fe15015d6e0 msgr2=0x7fe1500cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:52.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe156082700 1 --2- 172.21.15.3:0/2182069948 >> v2:172.21.15.32:3300/0 conn(0x7fe15015d6e0 0x7fe1500cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe156082700 1 -- 172.21.15.3:0/2182069948 >> v2:172.21.15.3:3300/0 conn(0x7fe1500cfd80 msgr2=0x7fe1500d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:52.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe156082700 1 --2- 172.21.15.3:0/2182069948 >> v2:172.21.15.3:3300/0 conn(0x7fe1500cfd80 0x7fe1500d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe156082700 1 -- 172.21.15.3:0/2182069948 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1500cd600 con 0x7fe150156ba0 2023-03-03T06:25:52.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.299+0000 7fe156883700 1 --2- 172.21.15.3:0/2182069948 >> v2:172.21.15.3:3300/0 conn(0x7fe1500cfd80 0x7fe1500d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:52.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.303+0000 7fe156082700 1 --2- 172.21.15.3:0/2182069948 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe150156ba0 0x7fe150071280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe1500d07b0 tx=0x7fe14c009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60b734d3b4ff2b16 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:52.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.303+0000 7fe155080700 1 -- 172.21.15.3:0/2182069948 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c011830 con 0x7fe150156ba0 2023-03-03T06:25:52.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.303+0000 7fe155080700 1 -- 172.21.15.3:0/2182069948 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14c011990 con 0x7fe150156ba0 2023-03-03T06:25:52.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.303+0000 7fe155080700 1 -- 172.21.15.3:0/2182069948 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c011d20 con 0x7fe150156ba0 2023-03-03T06:25:52.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.303+0000 7fe157084700 1 -- 172.21.15.3:0/2182069948 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe150156ba0 msgr2=0x7fe150071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.303+0000 7fe157084700 1 --2- 172.21.15.3:0/2182069948 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe150156ba0 0x7fe150071280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe1500d07b0 tx=0x7fe14c009b30 comp rx=0 tx=0).stop 2023-03-03T06:25:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.303+0000 7fe157084700 1 -- 172.21.15.3:0/2182069948 shutdown_connections 2023-03-03T06:25:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.303+0000 7fe157084700 1 --2- 172.21.15.3:0/2182069948 >> v2:172.21.15.3:3300/0 conn(0x7fe1500cfd80 0x7fe1500d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.303+0000 7fe157084700 1 --2- 172.21.15.3:0/2182069948 >> v2:172.21.15.32:3300/0 conn(0x7fe15015d6e0 0x7fe1500cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.303+0000 7fe157084700 1 --2- 172.21.15.3:0/2182069948 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe150156ba0 0x7fe150071280 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.303+0000 7fe157084700 1 -- 172.21.15.3:0/2182069948 >> 172.21.15.3:0/2182069948 conn(0x7fe15006e5c0 msgr2=0x7fe15006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:52.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.303+0000 7fe157084700 1 -- 172.21.15.3:0/2182069948 shutdown_connections 2023-03-03T06:25:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.303+0000 7fe157084700 1 -- 172.21.15.3:0/2182069948 wait complete. 2023-03-03T06:25:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.307+0000 7fe157084700 1 Processor -- start 2023-03-03T06:25:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.307+0000 7fe157084700 1 -- start start 2023-03-03T06:25:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe157084700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1500cfd80 0x7fe1501f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe157084700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe150156ba0 0x7fe1500b8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe157084700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15015d6e0 0x7fe150159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe157084700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500b3b60 con 0x7fe150156ba0 2023-03-03T06:25:52.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe157084700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500b49c0 con 0x7fe15015d6e0 2023-03-03T06:25:52.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe157084700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500b3db0 con 0x7fe1500cfd80 2023-03-03T06:25:52.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe155881700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe150156ba0 0x7fe1500b8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:52.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe155881700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe150156ba0 0x7fe1500b8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47932/0 (socket says 172.21.15.3:47932) 2023-03-03T06:25:52.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe155881700 1 -- 172.21.15.3:0/658644161 learned_addr learned my addr 172.21.15.3:0/658644161 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:52.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe156082700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1500cfd80 0x7fe1501f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:52.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe156883700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15015d6e0 0x7fe150159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe155881700 1 -- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1500cfd80 msgr2=0x7fe1501f2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe155881700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1500cfd80 0x7fe1501f2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe155881700 1 -- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15015d6e0 msgr2=0x7fe150159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe155881700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15015d6e0 0x7fe150159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe155881700 1 -- 172.21.15.3:0/658644161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe148009b30 con 0x7fe150156ba0 2023-03-03T06:25:52.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe155881700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe150156ba0 0x7fe1500b8150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe14000ea00 tx=0x7fe14000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:52.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe146ffd700 1 -- 172.21.15.3:0/658644161 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe140009720 con 0x7fe150156ba0 2023-03-03T06:25:52.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe146ffd700 1 -- 172.21.15.3:0/658644161 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14000c850 con 0x7fe150156ba0 2023-03-03T06:25:52.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe146ffd700 1 -- 172.21.15.3:0/658644161 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14000cbc0 con 0x7fe150156ba0 2023-03-03T06:25:52.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe157084700 1 -- 172.21.15.3:0/658644161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14c0094b0 con 0x7fe150156ba0 2023-03-03T06:25:52.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.311+0000 7fe157084700 1 -- 172.21.15.3:0/658644161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1500717c0 con 0x7fe150156ba0 2023-03-03T06:25:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.315+0000 7fe157084700 1 -- 172.21.15.3:0/658644161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1500cd8e0 con 0x7fe150156ba0 2023-03-03T06:25:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.315+0000 7fe146ffd700 1 -- 172.21.15.3:0/658644161 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe140019030 con 0x7fe150156ba0 2023-03-03T06:25:52.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.319+0000 7fe146ffd700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe134064ee0 0x7fe134067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:52.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.319+0000 7fe146ffd700 1 -- 172.21.15.3:0/658644161 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe1400822e0 con 0x7fe150156ba0 2023-03-03T06:25:52.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.319+0000 7fe156082700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe134064ee0 0x7fe134067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:52.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.323+0000 7fe156082700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe134064ee0 0x7fe134067390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe14c000c00 tx=0x7fe14c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:52.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.323+0000 7fe146ffd700 1 -- 172.21.15.3:0/658644161 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe140050040 con 0x7fe150156ba0 2023-03-03T06:25:52.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.503+0000 7fe157084700 1 -- 172.21.15.3:0/658644161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe1500cef40 con 0x7fe150156ba0 2023-03-03T06:25:52.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.503+0000 7fe146ffd700 1 -- 172.21.15.3:0/658644161 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe140053660 con 0x7fe150156ba0 2023-03-03T06:25:52.507 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:25:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.503+0000 7fe157084700 1 -- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe134064ee0 msgr2=0x7fe134067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.507+0000 7fe157084700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe134064ee0 0x7fe134067390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe14c000c00 tx=0x7fe14c008030 comp rx=0 tx=0).stop 2023-03-03T06:25:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.507+0000 7fe157084700 1 -- 172.21.15.3:0/658644161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe150156ba0 msgr2=0x7fe1500b8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.507+0000 7fe157084700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe150156ba0 0x7fe1500b8150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe14000ea00 tx=0x7fe14000edc0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.507+0000 7fe157084700 1 -- 172.21.15.3:0/658644161 shutdown_connections 2023-03-03T06:25:52.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.507+0000 7fe157084700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe134064ee0 0x7fe134067390 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.507+0000 7fe157084700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15015d6e0 0x7fe150159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.507+0000 7fe157084700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe150156ba0 0x7fe1500b8150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.507+0000 7fe157084700 1 --2- 172.21.15.3:0/658644161 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1500cfd80 0x7fe1501f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.507+0000 7fe157084700 1 -- 172.21.15.3:0/658644161 >> 172.21.15.3:0/658644161 conn(0x7fe15006e5c0 msgr2=0x7fe15006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:52.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.507+0000 7fe157084700 1 -- 172.21.15.3:0/658644161 shutdown_connections 2023-03-03T06:25:52.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.507+0000 7fe157084700 1 -- 172.21.15.3:0/658644161 wait complete. 2023-03-03T06:25:52.543 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-03T06:25:52.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.743+0000 7f78b6e12700 1 Processor -- start 2023-03-03T06:25:52.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.743+0000 7f78b6e12700 1 -- start start 2023-03-03T06:25:52.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.743+0000 7f78b6e12700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f78b0160880 0x7f78b00c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.743+0000 7f78b6e12700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78b00aea70 0x7f78b00ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:52.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.743+0000 7f78b6e12700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b00ce190 con 0x7f78b00adcf0 2023-03-03T06:25:52.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.743+0000 7f78b6e12700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b00ae190 con 0x7f78b0160880 2023-03-03T06:25:52.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.743+0000 7f78b6e12700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00c99d0 con 0x7f78b00aea70 2023-03-03T06:25:52.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b6611700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f78b00adcf0 0x7f78b01644f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46938/0 (socket says 172.21.15.3:46938) 2023-03-03T06:25:52.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b6611700 1 -- 172.21.15.3:0/2289376035 learned_addr learned my addr 172.21.15.3:0/2289376035 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:52.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b5e10700 1 --2- 172.21.15.3:0/2289376035 >> v2:172.21.15.32:3300/0 conn(0x7f78b0160880 0x7f78b00c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:52.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b560f700 1 --2- 172.21.15.3:0/2289376035 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78b00aea70 0x7f78b00ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:52.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b4e0e700 1 -- 172.21.15.3:0/2289376035 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810367865 0 0) 0x7f78b00ce190 con 0x7f78b00adcf0 2023-03-03T06:25:52.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b4e0e700 1 -- 172.21.15.3:0/2289376035 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f789c0035d0 con 0x7f78b00adcf0 2023-03-03T06:25:52.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b5e10700 1 -- 172.21.15.3:0/2289376035 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78b00aea70 msgr2=0x7f78b00ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:52.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b5e10700 1 --2- 172.21.15.3:0/2289376035 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78b00aea70 0x7f78b00ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b5e10700 1 -- 172.21.15.3:0/2289376035 >> v1:172.21.15.3:6789/0 conn(0x7f78b00adcf0 legacy=0x7f78b01644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:52.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b5e10700 1 -- 172.21.15.3:0/2289376035 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b00cd580 con 0x7f78b0160880 2023-03-03T06:25:52.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b5e10700 1 --2- 172.21.15.3:0/2289376035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b0160880 0x7f78b00c9dc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009850 tx=0x7f78ac009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed5609bfa1f188e server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:52.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b4e0e700 1 -- 172.21.15.3:0/2289376035 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a80019f0 con 0x7f78b0160880 2023-03-03T06:25:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b4e0e700 1 -- 172.21.15.3:0/2289376035 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ac010590 con 0x7f78b0160880 2023-03-03T06:25:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.747+0000 7f78b4e0e700 1 -- 172.21.15.3:0/2289376035 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac010940 con 0x7f78b0160880 2023-03-03T06:25:52.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.751+0000 7f78b6e12700 1 -- 172.21.15.3:0/2289376035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b0160880 msgr2=0x7f78b00c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:52.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.751+0000 7f78b6e12700 1 --2- 172.21.15.3:0/2289376035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b0160880 0x7f78b00c9dc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009850 tx=0x7f78ac009b60 comp rx=0 tx=0).stop 2023-03-03T06:25:52.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.751+0000 7f78b6e12700 1 -- 172.21.15.3:0/2289376035 shutdown_connections 2023-03-03T06:25:52.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.751+0000 7f78b6e12700 1 --2- 172.21.15.3:0/2289376035 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78b00aea70 0x7f78b00ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.751+0000 7f78b6e12700 1 --2- 172.21.15.3:0/2289376035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b0160880 0x7f78b00c9dc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.751+0000 7f78b6e12700 1 -- 172.21.15.3:0/2289376035 >> 172.21.15.3:0/2289376035 conn(0x7f78b0070a30 msgr2=0x7f78b0070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:52.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.751+0000 7f78b6e12700 1 -- 172.21.15.3:0/2289376035 shutdown_connections 2023-03-03T06:25:52.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.751+0000 7f78b6e12700 1 -- 172.21.15.3:0/2289376035 wait complete. 2023-03-03T06:25:52.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b6e12700 1 Processor -- start 2023-03-03T06:25:52.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b6e12700 1 -- start start 2023-03-03T06:25:52.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b6e12700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b00adcf0 0x7f78b01f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:52.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b6e12700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b00aea70 0x7f78b01f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:52.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b6e12700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78b0160880 0x7f78b00afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:52.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b6e12700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b01f6920 con 0x7f78b00aea70 2023-03-03T06:25:52.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b6e12700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00b2410 con 0x7f78b00adcf0 2023-03-03T06:25:52.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b6e12700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b01f62d0 con 0x7f78b0160880 2023-03-03T06:25:52.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b560f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b00aea70 0x7f78b01f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:52.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b560f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b00aea70 0x7f78b01f59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47936/0 (socket says 172.21.15.3:47936) 2023-03-03T06:25:52.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b560f700 1 -- 172.21.15.3:0/2234568466 learned_addr learned my addr 172.21.15.3:0/2234568466 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:52.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b6611700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78b0160880 0x7f78b00afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:52.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b5e10700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b00adcf0 0x7f78b01f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:52.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b560f700 1 -- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78b0160880 msgr2=0x7f78b00afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:52.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b560f700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78b0160880 0x7f78b00afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b560f700 1 -- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b00adcf0 msgr2=0x7f78b01f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:52.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b560f700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b00adcf0 0x7f78b01f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:52.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b560f700 1 -- 172.21.15.3:0/2234568466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac0094b0 con 0x7f78b00aea70 2023-03-03T06:25:52.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.755+0000 7f78b6611700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78b0160880 0x7f78b00afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:52.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.759+0000 7f78b560f700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b00aea70 0x7f78b01f59d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009fd0 tx=0x7f78a000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:52.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.759+0000 7f78a67fc700 1 -- 172.21.15.3:0/2234568466 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a00095a0 con 0x7f78b00aea70 2023-03-03T06:25:52.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.759+0000 7f78a67fc700 1 -- 172.21.15.3:0/2234568466 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a0013680 con 0x7f78b00aea70 2023-03-03T06:25:52.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.759+0000 7f78a67fc700 1 -- 172.21.15.3:0/2234568466 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0013a10 con 0x7f78b00aea70 2023-03-03T06:25:52.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.759+0000 7f78b6e12700 1 -- 172.21.15.3:0/2234568466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b00cdb40 con 0x7f78b00aea70 2023-03-03T06:25:52.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.759+0000 7f78b6e12700 1 -- 172.21.15.3:0/2234568466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b00cd860 con 0x7f78b00aea70 2023-03-03T06:25:52.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.759+0000 7f78b6e12700 1 -- 172.21.15.3:0/2234568466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b00cde20 con 0x7f78b00aea70 2023-03-03T06:25:52.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.763+0000 7f78a67fc700 1 -- 172.21.15.3:0/2234568466 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f78a0019030 con 0x7f78b00aea70 2023-03-03T06:25:52.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.763+0000 7f78a67fc700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7894064ee0 0x7f7894067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:52.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.767+0000 7f78a67fc700 1 -- 172.21.15.3:0/2234568466 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f78a0082080 con 0x7f78b00aea70 2023-03-03T06:25:52.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.767+0000 7f78b5e10700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7894064ee0 0x7f7894067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:52.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.767+0000 7f78b5e10700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7894064ee0 0x7f7894067390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f78ac013040 tx=0x7f78ac008000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:52.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.771+0000 7f78a67fc700 1 -- 172.21.15.3:0/2234568466 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f78a004fde0 con 0x7f78b00aea70 2023-03-03T06:25:52.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.987+0000 7f78b6e12700 1 -- 172.21.15.3:0/2234568466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78b0073e90 con 0x7f78b00aea70 2023-03-03T06:25:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.991+0000 7f78a67fc700 1 -- 172.21.15.3:0/2234568466 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4635 (secure 0 0 0) 0x7f78a0053400 con 0x7f78b00aea70 2023-03-03T06:25:52.994 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:25:52.994 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2744699972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2744699972},{"type":"v1","addr":"172.21.15.3:6834","nonce":2744699972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/708054272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":708054272},{"type":"v1","addr":"172.21.15.3:6835","nonce":708054272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2935923413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2935923413},{"type":"v1","addr":"172.21.15.32:6837","nonce":2935923413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-03T06:25:13.798672+0000","modified":"2023-03-03T06:25:41.755082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/691220172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":691220172},{"type":"v1","addr":"172.21.15.32:6836","nonce":691220172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-03-03T06:25:53.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.995+0000 7f78b6e12700 1 -- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7894064ee0 msgr2=0x7f7894067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.995+0000 7f78b6e12700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7894064ee0 0x7f7894067390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f78ac013040 tx=0x7f78ac008000 comp rx=0 tx=0).stop 2023-03-03T06:25:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.995+0000 7f78b6e12700 1 -- 172.21.15.3:0/2234568466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b00aea70 msgr2=0x7f78b01f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.995+0000 7f78b6e12700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b00aea70 0x7f78b01f59d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009fd0 tx=0x7f78a000eb20 comp rx=0 tx=0).stop 2023-03-03T06:25:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.995+0000 7f78b6e12700 1 -- 172.21.15.3:0/2234568466 shutdown_connections 2023-03-03T06:25:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.995+0000 7f78b6e12700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7894064ee0 0x7f7894067390 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.995+0000 7f78b6e12700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f78b0160880 0x7f78b00afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.995+0000 7f78b6e12700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78b00aea70 0x7f78b01f59d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.995+0000 7f78b6e12700 1 --2- 172.21.15.3:0/2234568466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78b00adcf0 0x7f78b01f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.995+0000 7f78b6e12700 1 -- 172.21.15.3:0/2234568466 >> 172.21.15.3:0/2234568466 conn(0x7f78b0070a30 msgr2=0x7f78b00aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:53.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.995+0000 7f78b6e12700 1 -- 172.21.15.3:0/2234568466 shutdown_connections 2023-03-03T06:25:53.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:52.995+0000 7f78b6e12700 1 -- 172.21.15.3:0/2234568466 wait complete. 2023-03-03T06:25:53.005 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2023-03-03T06:25:53.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-03-03T06:25:13.798672+0000', 'modified': '2023-03-03T06:25:41.755082+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54179}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54179': {'gid': 54179, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6836/691220172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 691220172}, {'type': 'v1', 'addr': '172.21.15.32:6836', 'nonce': 691220172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-03-03T06:25:53.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-03-03T06:25:53.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-03-03T06:25:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.651+0000 7fc4c4bef700 1 Processor -- start 2023-03-03T06:25:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.651+0000 7fc4c4bef700 1 -- start start 2023-03-03T06:25:53.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.655+0000 7fc4c4bef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4c00cfd30 0x7fc4c00d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.655+0000 7fc4c4bef700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c0160440 con 0x7fc4c01608d0 2023-03-03T06:25:53.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.655+0000 7fc4c4bef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c00cd3e0 con 0x7fc4c0153670 2023-03-03T06:25:53.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.655+0000 7fc4c4bef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c00cd6c0 con 0x7fc4c00cfd30 2023-03-03T06:25:53.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.655+0000 7fc4beffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc4c01608d0 0x7fc4c00cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46950/0 (socket says 172.21.15.3:46950) 2023-03-03T06:25:53.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.655+0000 7fc4beffd700 1 -- 172.21.15.3:0/2885988553 learned_addr learned my addr 172.21.15.3:0/2885988553 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:53.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.655+0000 7fc4bffff700 1 --2- 172.21.15.3:0/2885988553 >> v2:172.21.15.32:3300/0 conn(0x7fc4c00cfd30 0x7fc4c00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4be7fc700 1 -- 172.21.15.3:0/2885988553 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264727040 0 0) 0x7fc4c0160440 con 0x7fc4c01608d0 2023-03-03T06:25:53.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4be7fc700 1 -- 172.21.15.3:0/2885988553 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a80035d0 con 0x7fc4c01608d0 2023-03-03T06:25:53.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4bffff700 1 -- 172.21.15.3:0/2885988553 >> v1:172.21.15.32:6789/0 conn(0x7fc4c0153670 legacy=0x7fc4c0071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:53.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4be7fc700 1 -- 172.21.15.3:0/2885988553 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328415429 0 0) 0x7fc4c00cd3e0 con 0x7fc4c0153670 2023-03-03T06:25:53.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4bffff700 1 -- 172.21.15.3:0/2885988553 >> v1:172.21.15.3:6789/0 conn(0x7fc4c01608d0 legacy=0x7fc4c00cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:53.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4bffff700 1 -- 172.21.15.3:0/2885988553 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c01600d0 con 0x7fc4c00cfd30 2023-03-03T06:25:53.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4bffff700 1 --2- 172.21.15.3:0/2885988553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c00cfd30 0x7fc4c00d0b10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009dc0 tx=0x7fc4b800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b791d2b7aefddaad server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:53.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4be7fc700 1 -- 172.21.15.3:0/2885988553 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800e070 con 0x7fc4c00cfd30 2023-03-03T06:25:53.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4be7fc700 1 -- 172.21.15.3:0/2885988553 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b800bae0 con 0x7fc4c00cfd30 2023-03-03T06:25:53.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4be7fc700 1 -- 172.21.15.3:0/2885988553 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800be90 con 0x7fc4c00cfd30 2023-03-03T06:25:53.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4c4bef700 1 -- 172.21.15.3:0/2885988553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c00cfd30 msgr2=0x7fc4c00d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:53.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4c4bef700 1 --2- 172.21.15.3:0/2885988553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c00cfd30 0x7fc4c00d0b10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009dc0 tx=0x7fc4b800b330 comp rx=0 tx=0).stop 2023-03-03T06:25:53.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4c4bef700 1 -- 172.21.15.3:0/2885988553 shutdown_connections 2023-03-03T06:25:53.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4c4bef700 1 --2- 172.21.15.3:0/2885988553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c00cfd30 0x7fc4c00d0b10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.659+0000 7fc4c4bef700 1 -- 172.21.15.3:0/2885988553 >> 172.21.15.3:0/2885988553 conn(0x7fc4c006e5c0 msgr2=0x7fc4c006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:53.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.663+0000 7fc4c4bef700 1 -- 172.21.15.3:0/2885988553 shutdown_connections 2023-03-03T06:25:53.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.663+0000 7fc4c4bef700 1 -- 172.21.15.3:0/2885988553 wait complete. 2023-03-03T06:25:53.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.663+0000 7fc4c4bef700 1 Processor -- start 2023-03-03T06:25:53.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.667+0000 7fc4c4bef700 1 -- start start 2023-03-03T06:25:53.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.667+0000 7fc4c4bef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c00cfd30 0x7fc4c01f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.667+0000 7fc4c4bef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c0153670 0x7fc4c01f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.667+0000 7fc4c4bef700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4c01608d0 0x7fc4c00c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.667+0000 7fc4c4bef700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c00b5190 con 0x7fc4c01608d0 2023-03-03T06:25:53.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.667+0000 7fc4c4bef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c01fa420 con 0x7fc4c0153670 2023-03-03T06:25:53.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.667+0000 7fc4c4bef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c01f5e00 con 0x7fc4c00cfd30 2023-03-03T06:25:53.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.667+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4c01608d0 0x7fc4c00c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.667+0000 7fc4bf7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c00cfd30 0x7fc4c01f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.667+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4c01608d0 0x7fc4c00c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47944/0 (socket says 172.21.15.3:47944) 2023-03-03T06:25:53.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.667+0000 7fc4bffff700 1 -- 172.21.15.3:0/3262927980 learned_addr learned my addr 172.21.15.3:0/3262927980 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:53.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.667+0000 7fc4beffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c0153670 0x7fc4c01f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.671+0000 7fc4bffff700 1 -- 172.21.15.3:0/3262927980 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c00cfd30 msgr2=0x7fc4c01f9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:53.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.671+0000 7fc4bffff700 1 --2- 172.21.15.3:0/3262927980 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c00cfd30 0x7fc4c01f9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.671+0000 7fc4bffff700 1 -- 172.21.15.3:0/3262927980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c0153670 msgr2=0x7fc4c01f58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:53.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.671+0000 7fc4bffff700 1 --2- 172.21.15.3:0/3262927980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c0153670 0x7fc4c01f58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.671+0000 7fc4bffff700 1 -- 172.21.15.3:0/3262927980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b8009a70 con 0x7fc4c01608d0 2023-03-03T06:25:53.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.671+0000 7fc4bffff700 1 --2- 172.21.15.3:0/3262927980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4c01608d0 0x7fc4c00c8ce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8013000 tx=0x7fc4b8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:53.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.671+0000 7fc4a7fff700 1 -- 172.21.15.3:0/3262927980 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8027480 con 0x7fc4c01608d0 2023-03-03T06:25:53.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.671+0000 7fc4a7fff700 1 -- 172.21.15.3:0/3262927980 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8020070 con 0x7fc4c01608d0 2023-03-03T06:25:53.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.671+0000 7fc4a7fff700 1 -- 172.21.15.3:0/3262927980 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8027840 con 0x7fc4c01608d0 2023-03-03T06:25:53.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.671+0000 7fc4c4bef700 1 -- 172.21.15.3:0/3262927980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c00cdb30 con 0x7fc4c01608d0 2023-03-03T06:25:53.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.671+0000 7fc4c4bef700 1 -- 172.21.15.3:0/3262927980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c00cd9a0 con 0x7fc4c01608d0 2023-03-03T06:25:53.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.679+0000 7fc4a7fff700 1 -- 172.21.15.3:0/3262927980 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc4b802e070 con 0x7fc4c01608d0 2023-03-03T06:25:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.679+0000 7fc4a7fff700 1 --2- 172.21.15.3:0/3262927980 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc4ac064ee0 0x7fc4ac067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4a7fff700 1 -- 172.21.15.3:0/3262927980 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc4b8022070 con 0x7fc4c01608d0 2023-03-03T06:25:53.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4c90db740 1 Processor -- start 2023-03-03T06:25:53.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4bf7fe700 1 --2- 172.21.15.3:0/3262927980 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc4ac064ee0 0x7fc4ac067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4c90db740 1 -- start start 2023-03-03T06:25:53.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4c90db740 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x13f5ae0 0x16acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4c90db740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5710 0x16ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4c90db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1440790 0x1661340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4c90db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x1564f00 con 0x1440790 2023-03-03T06:25:53.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4c90db740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x15d4ff0 con 0x13f5710 2023-03-03T06:25:53.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4c90db740 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x15ce2f0 con 0x13f5ae0 2023-03-03T06:25:53.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1440790 0x1661340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1440790 0x1661340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47952/0 (socket says 172.21.15.3:47952) 2023-03-03T06:25:53.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4bffff700 1 -- 172.21.15.3:0/3561465926 learned_addr learned my addr 172.21.15.3:0/3561465926 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:53.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4bf7fe700 1 --2- 172.21.15.3:0/3262927980 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc4ac064ee0 0x7fc4ac067390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0007fd0 tx=0x7fc4b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:53.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4beffd700 1 --2- 172.21.15.3:0/3561465926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5710 0x16ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4bf7fe700 1 --2- 172.21.15.3:0/3561465926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x13f5ae0 0x16acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4bffff700 1 -- 172.21.15.3:0/3561465926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x13f5ae0 msgr2=0x16acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:53.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4bffff700 1 --2- 172.21.15.3:0/3561465926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x13f5ae0 0x16acd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4bffff700 1 -- 172.21.15.3:0/3561465926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5710 msgr2=0x16ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:53.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4bffff700 1 --2- 172.21.15.3:0/3561465926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5710 0x16ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4bffff700 1 -- 172.21.15.3:0/3561465926 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x15f3c90 con 0x1440790 2023-03-03T06:25:53.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.683+0000 7fc4bf7fe700 1 --2- 172.21.15.3:0/3561465926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x13f5ae0 0x16acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:25:53.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.687+0000 7fc4bffff700 1 --2- 172.21.15.3:0/3561465926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1440790 0x1661340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc4b805da70 tx=0x7fc4b805daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe01676124529d5e server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:53.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.687+0000 7fc4a5ffb700 1 -- 172.21.15.3:0/3561465926 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80516d0 con 0x1440790 2023-03-03T06:25:53.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.687+0000 7fc4a5ffb700 1 -- 172.21.15.3:0/3561465926 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8051830 con 0x1440790 2023-03-03T06:25:53.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.687+0000 7fc4a5ffb700 1 -- 172.21.15.3:0/3561465926 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8054840 con 0x1440790 2023-03-03T06:25:53.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.687+0000 7fc4c90db740 1 -- 172.21.15.3:0/3561465926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1440790 msgr2=0x1661340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:53.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.687+0000 7fc4c90db740 1 --2- 172.21.15.3:0/3561465926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1440790 0x1661340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc4b805da70 tx=0x7fc4b805daa0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.687+0000 7fc4c90db740 1 -- 172.21.15.3:0/3561465926 shutdown_connections 2023-03-03T06:25:53.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.687+0000 7fc4c90db740 1 --2- 172.21.15.3:0/3561465926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1440790 0x1661340 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.687+0000 7fc4c90db740 1 --2- 172.21.15.3:0/3561465926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5710 0x16ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.687+0000 7fc4c90db740 1 --2- 172.21.15.3:0/3561465926 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x13f5ae0 0x16acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.687+0000 7fc4c90db740 1 -- 172.21.15.3:0/3561465926 >> 172.21.15.3:0/3561465926 conn(0x13f5eb0 msgr2=0x168e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:53.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.687+0000 7fc4c90db740 1 -- 172.21.15.3:0/3561465926 shutdown_connections 2023-03-03T06:25:53.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.687+0000 7fc4c90db740 1 -- 172.21.15.3:0/3561465926 wait complete. 2023-03-03T06:25:53.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.691+0000 7fc4c90db740 1 -- start start 2023-03-03T06:25:53.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.691+0000 7fc4c90db740 1 Processor -- start 2023-03-03T06:25:53.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.691+0000 7fc4c90db740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5710 0x16a3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.691+0000 7fc4c90db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5ae0 0x168e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.691+0000 7fc4c90db740 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x1440790 0x168ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.691+0000 7fc4c90db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x147ab20 con 0x13f5ae0 2023-03-03T06:25:53.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.691+0000 7fc4c90db740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x16a3540 con 0x13f5710 2023-03-03T06:25:53.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.691+0000 7fc4c90db740 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x16b2ae0 con 0x1440790 2023-03-03T06:25:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.691+0000 7fc4c90db740 10 client.0 _ll_register_callbacks cb 0x1629d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-03T06:25:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4beffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5ae0 0x168e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4beffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5ae0 0x168e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47960/0 (socket says 172.21.15.3:47960) 2023-03-03T06:25:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4beffd700 1 -- 172.21.15.3:0/2079259022 learned_addr learned my addr 172.21.15.3:0/2079259022 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4bffff700 1 --2- 172.21.15.3:0/2079259022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x1440790 0x168ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4bf7fe700 1 --2- 172.21.15.3:0/2079259022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5710 0x16a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4beffd700 1 -- 172.21.15.3:0/2079259022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x1440790 msgr2=0x168ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4beffd700 1 --2- 172.21.15.3:0/2079259022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x1440790 0x168ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4beffd700 1 -- 172.21.15.3:0/2079259022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5710 msgr2=0x16a3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4beffd700 1 --2- 172.21.15.3:0/2079259022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5710 0x16a3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4beffd700 1 -- 172.21.15.3:0/2079259022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c00cd9a0 con 0x13f5ae0 2023-03-03T06:25:53.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4beffd700 1 --2- 172.21.15.3:0/2079259022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5ae0 0x168e690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4000f60 tx=0x7fc4b4007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f7d02129273d8 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:53.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc49effd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-03-03T06:25:53.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc49effd700 1 -- 172.21.15.3:0/2079259022 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4001a10 con 0x13f5ae0 2023-03-03T06:25:53.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc49effd700 1 -- 172.21.15.3:0/2079259022 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc4b4001b70 con 0x13f5ae0 2023-03-03T06:25:53.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4c90db740 1 -- 172.21.15.3:0/2079259022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x15c7370 con 0x13f5ae0 2023-03-03T06:25:53.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc49effd700 1 -- 172.21.15.3:0/2079259022 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4b4001d30 con 0x13f5ae0 2023-03-03T06:25:53.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4c90db740 10 client.4803 fetch_fsmap learned FSMap version 8 2023-03-03T06:25:53.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4c90db740 1 -- 172.21.15.3:0/2079259022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x14419e0 con 0x13f5ae0 2023-03-03T06:25:53.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc49effd700 1 -- 172.21.15.3:0/2079259022 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x15c7370 con 0x13f5ae0 2023-03-03T06:25:53.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc49effd700 1 -- 172.21.15.3:0/2079259022 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc4b4015470 con 0x13f5ae0 2023-03-03T06:25:53.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4c90db740 10 client.4803 fetch_fsmap finished waiting for FSMap version 8 2023-03-03T06:25:53.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4c90db740 10 client.4803 resolve_mds: resolved name 'b' to daemon mds.b 2023-03-03T06:25:53.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4c90db740 20 client.4803 populate_metadata read hostname 'smithi003' 2023-03-03T06:25:53.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2079259022 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x16773f0 0x1675280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4c90db740 4 client.4803 mds_command: new command op to 54179 tid=0[{"prefix": "get_command_descriptions"}] 2023-03-03T06:25:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.695+0000 7fc4c90db740 1 -- 172.21.15.3:0/2079259022 --> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x15c7090 con 0x16773f0 2023-03-03T06:25:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.699+0000 7fc4bf7fe700 1 --2- 172.21.15.3:0/2079259022 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x16773f0 0x1675280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.699+0000 7fc4bf7fe700 1 --2- 172.21.15.3:0/2079259022 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x16773f0 0x1675280 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d979b4b224beced server_cookie=e072df9b29a4ac2b in_seq=0 out_seq=0 2023-03-03T06:25:53.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.699+0000 7fc49effd700 10 client.4803 ms_handle_connect on v2:172.21.15.32:6834/691220172 2023-03-03T06:25:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.699+0000 7fc49effd700 1 -- 172.21.15.3:0/2079259022 <== mds.0 v2:172.21.15.32:6834/691220172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18278 (crc 0 0 0) 0x15c7090 con 0x16773f0 2023-03-03T06:25:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.699+0000 7fc49effd700 10 client.4803 handle_command_reply: tid=0 2023-03-03T06:25:53.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.699+0000 7fc4c90db740 1 client.4803 shutdown 2023-03-03T06:25:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.703+0000 7fc4c90db740 1 -- 172.21.15.3:0/2079259022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5ae0 msgr2=0x168e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:53.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.703+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2079259022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5ae0 0x168e690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4000f60 tx=0x7fc4b4007860 comp rx=0 tx=0).stop 2023-03-03T06:25:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.703+0000 7fc4c90db740 1 -- 172.21.15.3:0/2079259022 shutdown_connections 2023-03-03T06:25:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.703+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2079259022 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x16773f0 0x1675280 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.703+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2079259022 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x1440790 0x168ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.703+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2079259022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5ae0 0x168e690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.703+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2079259022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5710 0x16a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.703+0000 7fc4c90db740 1 -- 172.21.15.3:0/2079259022 >> 172.21.15.3:0/2079259022 conn(0x13f5eb0 msgr2=0x168a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:53.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.703+0000 7fc49effd700 0 client.4803 ms_handle_reset on v2:172.21.15.32:6834/691220172 2023-03-03T06:25:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.703+0000 7fc4c90db740 1 -- 172.21.15.3:0/2079259022 shutdown_connections 2023-03-03T06:25:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.703+0000 7fc4c90db740 1 -- 172.21.15.3:0/2079259022 wait complete. 2023-03-03T06:25:53.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.703+0000 7fc4c90db740 20 client.4803 trim_cache size 0 max 16384 2023-03-03T06:25:53.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.723+0000 7fc4c90db740 1 Processor -- start 2023-03-03T06:25:53.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.723+0000 7fc4c90db740 1 -- start start 2023-03-03T06:25:53.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.723+0000 7fc4c90db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5710 0x16a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.723+0000 7fc4c90db740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5ae0 0x16a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4c90db740 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x1440790 0x16a2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4c90db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x16754a0 con 0x13f5710 2023-03-03T06:25:53.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4c90db740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x165c480 con 0x13f5ae0 2023-03-03T06:25:53.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4c90db740 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x15a3350 con 0x1440790 2023-03-03T06:25:53.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x1440790 0x16a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4bf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5710 0x16a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x1440790 0x16a2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:56486/0 (socket says 172.21.15.3:56486) 2023-03-03T06:25:53.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4bffff700 1 -- 172.21.15.3:0/2101148937 learned_addr learned my addr 172.21.15.3:0/2101148937 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:53.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4beffd700 1 --2- 172.21.15.3:0/2101148937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5ae0 0x16a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4bffff700 1 -- 172.21.15.3:0/2101148937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5ae0 msgr2=0x16a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:53.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4bffff700 1 --2- 172.21.15.3:0/2101148937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5ae0 0x16a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4bffff700 1 -- 172.21.15.3:0/2101148937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5710 msgr2=0x16a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:53.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4bffff700 1 --2- 172.21.15.3:0/2101148937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5710 0x16a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4bffff700 1 -- 172.21.15.3:0/2101148937 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c014e5e0 con 0x1440790 2023-03-03T06:25:53.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4bf7fe700 1 --2- 172.21.15.3:0/2101148937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5710 0x16a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:53.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4bffff700 1 --2- 172.21.15.3:0/2101148937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x1440790 0x16a2070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80555e0 tx=0x7fc4b802c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69aa7ee04a370161 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:53.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4a67fc700 1 -- 172.21.15.3:0/2101148937 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8051c20 con 0x1440790 2023-03-03T06:25:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4a67fc700 1 -- 172.21.15.3:0/2101148937 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8053d40 con 0x1440790 2023-03-03T06:25:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4a67fc700 1 -- 172.21.15.3:0/2101148937 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b805d970 con 0x1440790 2023-03-03T06:25:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4c90db740 1 -- 172.21.15.3:0/2101148937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x1440790 msgr2=0x16a2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2101148937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x1440790 0x16a2070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80555e0 tx=0x7fc4b802c000 comp rx=0 tx=0).stop 2023-03-03T06:25:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4c90db740 1 -- 172.21.15.3:0/2101148937 shutdown_connections 2023-03-03T06:25:53.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2101148937 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x1440790 0x16a2070 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2101148937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x13f5ae0 0x16a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2101148937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5710 0x16a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4c90db740 1 -- 172.21.15.3:0/2101148937 >> 172.21.15.3:0/2101148937 conn(0x13f5eb0 msgr2=0x16758e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:53.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4c90db740 1 -- 172.21.15.3:0/2101148937 shutdown_connections 2023-03-03T06:25:53.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.727+0000 7fc4c90db740 1 -- 172.21.15.3:0/2101148937 wait complete. 2023-03-03T06:25:53.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4c90db740 1 -- start start 2023-03-03T06:25:53.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4c90db740 1 Processor -- start 2023-03-03T06:25:53.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4c90db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5710 0x170de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4c90db740 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x13f5ae0 0x1663f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4c90db740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1440790 0x16c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4c90db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x170e340 con 0x13f5710 2023-03-03T06:25:53.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4c90db740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x16a2a70 con 0x1440790 2023-03-03T06:25:53.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4c90db740 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x1666b60 con 0x13f5ae0 2023-03-03T06:25:53.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4c90db740 10 client.0 _ll_register_callbacks cb 0x15cd080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-03T06:25:53.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4bf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5710 0x170de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4bf7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5710 0x170de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47988/0 (socket says 172.21.15.3:47988) 2023-03-03T06:25:53.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4bf7fe700 1 -- 172.21.15.3:0/2073574457 learned_addr learned my addr 172.21.15.3:0/2073574457 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:53.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4beffd700 1 --2- 172.21.15.3:0/2073574457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x13f5ae0 0x1663f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.731+0000 7fc4bffff700 1 --2- 172.21.15.3:0/2073574457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1440790 0x16c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4bf7fe700 1 -- 172.21.15.3:0/2073574457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x13f5ae0 msgr2=0x1663f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:53.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4bf7fe700 1 --2- 172.21.15.3:0/2073574457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x13f5ae0 0x1663f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4bf7fe700 1 -- 172.21.15.3:0/2073574457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1440790 msgr2=0x16c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:53.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4bf7fe700 1 --2- 172.21.15.3:0/2073574457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1440790 0x16c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4bf7fe700 1 -- 172.21.15.3:0/2073574457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b000f690 con 0x13f5710 2023-03-03T06:25:53.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4bf7fe700 1 --2- 172.21.15.3:0/2073574457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5710 0x170de00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0000f60 tx=0x7fc4b0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87550c7c23fb6634 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:53.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc49effd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-03-03T06:25:53.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc49effd700 1 -- 172.21.15.3:0/2073574457 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0022a30 con 0x13f5710 2023-03-03T06:25:53.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4c90db740 1 -- 172.21.15.3:0/2073574457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4b8051830 con 0x13f5710 2023-03-03T06:25:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc49effd700 1 -- 172.21.15.3:0/2073574457 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc4b0022b90 con 0x13f5710 2023-03-03T06:25:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc49effd700 1 -- 172.21.15.3:0/2073574457 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4b0022d50 con 0x13f5710 2023-03-03T06:25:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4c90db740 10 client.4809 fetch_fsmap learned FSMap version 8 2023-03-03T06:25:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4c90db740 1 -- 172.21.15.3:0/2073574457 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x15f3c90 con 0x13f5710 2023-03-03T06:25:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc49effd700 1 -- 172.21.15.3:0/2073574457 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8051830 con 0x13f5710 2023-03-03T06:25:53.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc49effd700 1 -- 172.21.15.3:0/2073574457 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc4b0033100 con 0x13f5710 2023-03-03T06:25:53.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4c90db740 10 client.4809 fetch_fsmap finished waiting for FSMap version 8 2023-03-03T06:25:53.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4c90db740 10 client.4809 resolve_mds: resolved name 'b' to daemon mds.b 2023-03-03T06:25:53.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4c90db740 20 client.4809 populate_metadata read hostname 'smithi003' 2023-03-03T06:25:53.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2073574457 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x1661c60 0x1662030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:53.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4c90db740 4 client.4809 mds_command: new command op to 54179 tid=0[{"prefix": "status"}] 2023-03-03T06:25:53.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.735+0000 7fc4c90db740 1 -- 172.21.15.3:0/2073574457 --> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1472260 con 0x1661c60 2023-03-03T06:25:53.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc4beffd700 1 --2- 172.21.15.3:0/2073574457 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x1661c60 0x1662030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:53.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc4beffd700 1 --2- 172.21.15.3:0/2073574457 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x1661c60 0x1662030 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=992a2ec6dc23c32d server_cookie=e4f3201197b3f68a in_seq=0 out_seq=0 2023-03-03T06:25:53.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc49effd700 10 client.4809 ms_handle_connect on v2:172.21.15.32:6834/691220172 2023-03-03T06:25:53.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc49effd700 1 -- 172.21.15.3:0/2073574457 <== mds.0 v2:172.21.15.32:6834/691220172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x1472260 con 0x1661c60 2023-03-03T06:25:53.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc49effd700 10 client.4809 handle_command_reply: tid=0 2023-03-03T06:25:53.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc4c90db740 1 client.4809 shutdown 2023-03-03T06:25:53.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc4c90db740 1 -- 172.21.15.3:0/2073574457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5710 msgr2=0x170de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:53.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2073574457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5710 0x170de00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0000f60 tx=0x7fc4b0000ed0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc4c90db740 1 -- 172.21.15.3:0/2073574457 shutdown_connections 2023-03-03T06:25:53.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2073574457 >> [v2:172.21.15.32:6834/691220172,v1:172.21.15.32:6836/691220172] conn(0x1661c60 0x1662030 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2073574457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x1440790 0x16c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2073574457 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x13f5ae0 0x1663f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc4c90db740 1 --2- 172.21.15.3:0/2073574457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13f5710 0x170de00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc4c90db740 1 -- 172.21.15.3:0/2073574457 >> 172.21.15.3:0/2073574457 conn(0x13f5eb0 msgr2=0x168ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:53.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc49effd700 0 client.4809 ms_handle_reset on v2:172.21.15.32:6834/691220172 2023-03-03T06:25:53.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc4c90db740 1 -- 172.21.15.3:0/2073574457 shutdown_connections 2023-03-03T06:25:53.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.739+0000 7fc4c90db740 1 -- 172.21.15.3:0/2073574457 wait complete. 2023-03-03T06:25:53.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.743+0000 7fc4c90db740 20 client.4809 trim_cache size 0 max 16384 2023-03-03T06:25:53.747 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-03-03T06:25:53.748 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "7d496254-7268-405f-a1d8-c8033dc8659a", 2023-03-03T06:25:53.748 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-03-03T06:25:53.748 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 54179, 2023-03-03T06:25:53.748 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-03-03T06:25:53.748 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-03-03T06:25:53.748 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-03-03T06:25:53.749 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 39.932907692999997, 2023-03-03T06:25:53.749 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 8, 2023-03-03T06:25:53.749 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 29, 2023-03-03T06:25:53.749 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 28, 2023-03-03T06:25:53.749 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 46.330961354999999 2023-03-03T06:25:53.749 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-03-03T06:25:53.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.743+0000 7fc4c4bef700 1 -- 172.21.15.3:0/3262927980 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc4ac064ee0 msgr2=0x7fc4ac067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:53.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.743+0000 7fc4c4bef700 1 --2- 172.21.15.3:0/3262927980 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc4ac064ee0 0x7fc4ac067390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0007fd0 tx=0x7fc4b0009040 comp rx=0 tx=0).stop 2023-03-03T06:25:53.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.743+0000 7fc4c4bef700 1 -- 172.21.15.3:0/3262927980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4c01608d0 msgr2=0x7fc4c00c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:53.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.743+0000 7fc4c4bef700 1 --2- 172.21.15.3:0/3262927980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4c01608d0 0x7fc4c00c8ce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8013000 tx=0x7fc4b8004af0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.743+0000 7fc4c4bef700 1 -- 172.21.15.3:0/3262927980 shutdown_connections 2023-03-03T06:25:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.743+0000 7fc4c4bef700 1 --2- 172.21.15.3:0/3262927980 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc4ac064ee0 0x7fc4ac067390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.743+0000 7fc4c4bef700 1 --2- 172.21.15.3:0/3262927980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4c01608d0 0x7fc4c00c8ce0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.743+0000 7fc4c4bef700 1 --2- 172.21.15.3:0/3262927980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c0153670 0x7fc4c01f58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.743+0000 7fc4c4bef700 1 --2- 172.21.15.3:0/3262927980 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4c00cfd30 0x7fc4c01f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.743+0000 7fc4c4bef700 1 -- 172.21.15.3:0/3262927980 >> 172.21.15.3:0/3262927980 conn(0x7fc4c006e5c0 msgr2=0x7fc4c006ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.743+0000 7fc4c4bef700 1 -- 172.21.15.3:0/3262927980 shutdown_connections 2023-03-03T06:25:53.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:53.743+0000 7fc4c4bef700 1 -- 172.21.15.3:0/3262927980 wait complete. 2023-03-03T06:25:53.779 INFO:teuthology.run_tasks:Running task kclient... 2023-03-03T06:25:53.824 INFO:tasks.kclient:Mounting kernel clients... 2023-03-03T06:25:53.825 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-03-03T06:25:53.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:53.825 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-03-03T06:25:53.848 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-03-03T06:25:53.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:53.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-03-03T06:25:53.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:53.934 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2023-03-03T06:25:54.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:54.128 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-03-03T06:25:54.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:54.148 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2023-03-03T06:25:54.239 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2023-03-03T06:25:54.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-03T06:25:54.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:54.244 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-03-03T06:25:54.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:54.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-03-03T06:25:54.362 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2023-03-03T06:25:54.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-03T06:25:54.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:54.369 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-03-03T06:25:54.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:54.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-03-03T06:25:54.529 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2023-03-03T06:25:54.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-03T06:25:54.536 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-03-03T06:25:54.537 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-03-03T06:25:54.537 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-03-03T06:25:54.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-03-03T06:25:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.755+0000 7f4cfdff9700 1 Processor -- start 2023-03-03T06:25:54.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.755+0000 7f4cfdff9700 1 -- start start 2023-03-03T06:25:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.755+0000 7f4cfdff9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cf80aee70 0x7f4cf8158440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.755+0000 7f4cfdff9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cf815c040 0x7f4cf815c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.755+0000 7f4cfdff9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf80cdd30 con 0x7f4cf806a780 2023-03-03T06:25:54.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.755+0000 7f4cfdff9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf815ca20 con 0x7f4cf815c040 2023-03-03T06:25:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.755+0000 7f4cfdff9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf8158980 con 0x7f4cf80aee70 2023-03-03T06:25:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.755+0000 7f4cf7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cf80aee70 0x7f4cf8158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.755+0000 7f4cf7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cf80aee70 0x7f4cf8158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:56510/0 (socket says 172.21.15.3:56510) 2023-03-03T06:25:54.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.755+0000 7f4cfd7f8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4cf815c040 0x7f4cf815c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.755+0000 7f4cf7fff700 1 -- 172.21.15.3:0/878249533 learned_addr learned my addr 172.21.15.3:0/878249533 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cf7fff700 1 -- 172.21.15.3:0/878249533 >> v2:172.21.15.32:3300/0 conn(0x7f4cf815c040 msgr2=0x7f4cf815c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cf7fff700 1 --2- 172.21.15.3:0/878249533 >> v2:172.21.15.32:3300/0 conn(0x7f4cf815c040 0x7f4cf815c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:54.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cf7fff700 1 -- 172.21.15.3:0/878249533 >> v1:172.21.15.32:6789/0 conn(0x7f4cf806a780 legacy=0x7f4cf80c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cf7fff700 1 -- 172.21.15.3:0/878249533 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf80ca4d0 con 0x7f4cf80aee70 2023-03-03T06:25:54.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cf7fff700 1 --2- 172.21.15.3:0/878249533 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cf80aee70 0x7f4cf8158440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009910 tx=0x7f4ce4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18b83dcc6ce1dd9f server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:54.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cf77fe700 1 -- 172.21.15.3:0/878249533 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4011830 con 0x7f4cf80aee70 2023-03-03T06:25:54.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cf77fe700 1 -- 172.21.15.3:0/878249533 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce4011990 con 0x7f4cf80aee70 2023-03-03T06:25:54.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cf77fe700 1 -- 172.21.15.3:0/878249533 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4011d40 con 0x7f4cf80aee70 2023-03-03T06:25:54.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cfdff9700 1 -- 172.21.15.3:0/878249533 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cf80aee70 msgr2=0x7f4cf8158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:54.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cfdff9700 1 --2- 172.21.15.3:0/878249533 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cf80aee70 0x7f4cf8158440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009910 tx=0x7f4ce4009c20 comp rx=0 tx=0).stop 2023-03-03T06:25:54.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cfdff9700 1 -- 172.21.15.3:0/878249533 shutdown_connections 2023-03-03T06:25:54.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cfdff9700 1 --2- 172.21.15.3:0/878249533 >> v2:172.21.15.32:3300/0 conn(0x7f4cf815c040 0x7f4cf815c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:54.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cfdff9700 1 --2- 172.21.15.3:0/878249533 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cf80aee70 0x7f4cf8158440 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:54.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cfdff9700 1 -- 172.21.15.3:0/878249533 >> 172.21.15.3:0/878249533 conn(0x7f4cf8070a30 msgr2=0x7f4cf8070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:54.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cfdff9700 1 -- 172.21.15.3:0/878249533 shutdown_connections 2023-03-03T06:25:54.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.759+0000 7f4cfdff9700 1 -- 172.21.15.3:0/878249533 wait complete. 2023-03-03T06:25:54.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.763+0000 7f4cfdff9700 1 Processor -- start 2023-03-03T06:25:54.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfdff9700 1 -- start start 2023-03-03T06:25:54.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfdff9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf806a780 0x7f4cf81650a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:54.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfdff9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cf80aee70 0x7f4cf815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:54.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfdff9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cf815c040 0x7f4cf81592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:54.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfdff9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf814cc20 con 0x7f4cf815c040 2023-03-03T06:25:54.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfdff9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf8165760 con 0x7f4cf806a780 2023-03-03T06:25:54.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfdff9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf814c5d0 con 0x7f4cf80aee70 2023-03-03T06:25:54.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfcff7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf806a780 0x7f4cf81650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:54.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfcff7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf806a780 0x7f4cf81650a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:44900/0 (socket says 172.21.15.3:44900) 2023-03-03T06:25:54.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cf7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cf80aee70 0x7f4cf815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:54.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfcff7700 1 -- 172.21.15.3:0/3900840782 learned_addr learned my addr 172.21.15.3:0/3900840782 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:54.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfd7f8700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cf815c040 0x7f4cf81592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:54.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfd7f8700 1 -- 172.21.15.3:0/3900840782 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cf80aee70 msgr2=0x7f4cf815b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:54.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfd7f8700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cf80aee70 0x7f4cf815b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:54.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfd7f8700 1 -- 172.21.15.3:0/3900840782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf806a780 msgr2=0x7f4cf81650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:54.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfd7f8700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf806a780 0x7f4cf81650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:54.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cfd7f8700 1 -- 172.21.15.3:0/3900840782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce4009570 con 0x7f4cf815c040 2023-03-03T06:25:54.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.767+0000 7f4cf7fff700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cf80aee70 0x7f4cf815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:25:54.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.771+0000 7f4cfd7f8700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cf815c040 0x7f4cf81592c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000eba0 tx=0x7f4cf000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:54.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.771+0000 7f4cf57fa700 1 -- 172.21.15.3:0/3900840782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0009a80 con 0x7f4cf815c040 2023-03-03T06:25:54.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.771+0000 7f4cf57fa700 1 -- 172.21.15.3:0/3900840782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf000f040 con 0x7f4cf815c040 2023-03-03T06:25:54.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.771+0000 7f4cf57fa700 1 -- 172.21.15.3:0/3900840782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf000cc40 con 0x7f4cf815c040 2023-03-03T06:25:54.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.771+0000 7f4cfdff9700 1 -- 172.21.15.3:0/3900840782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf80cd9c0 con 0x7f4cf815c040 2023-03-03T06:25:54.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.771+0000 7f4cfdff9700 1 -- 172.21.15.3:0/3900840782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf80cd6e0 con 0x7f4cf815c040 2023-03-03T06:25:54.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.771+0000 7f4cfdff9700 1 -- 172.21.15.3:0/3900840782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf80ca7b0 con 0x7f4cf815c040 2023-03-03T06:25:54.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.775+0000 7f4cf57fa700 1 -- 172.21.15.3:0/3900840782 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4cf0014480 con 0x7f4cf815c040 2023-03-03T06:25:54.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.779+0000 7f4cf57fa700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4cdc064ee0 0x7f4cdc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:54.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.779+0000 7f4cf57fa700 1 -- 172.21.15.3:0/3900840782 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4cf0082d20 con 0x7f4cf815c040 2023-03-03T06:25:54.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.779+0000 7f4cfcff7700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4cdc064ee0 0x7f4cdc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:54.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.779+0000 7f4cfcff7700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4cdc064ee0 0x7f4cdc067390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4cec001940 tx=0x7f4cec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:54.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:54.783+0000 7f4cf57fa700 1 -- 172.21.15.3:0/3900840782 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4cf0050a40 con 0x7f4cf815c040 2023-03-03T06:25:55.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.003+0000 7f4cfdff9700 1 -- 172.21.15.3:0/3900840782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4cf814c430 con 0x7f4cf815c040 2023-03-03T06:25:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.007+0000 7f4cf57fa700 1 -- 172.21.15.3:0/3900840782 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4cf814c430 con 0x7f4cf815c040 2023-03-03T06:25:55.010 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-03-03T06:25:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.007+0000 7f4cfdff9700 1 -- 172.21.15.3:0/3900840782 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4cdc064ee0 msgr2=0x7f4cdc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.007+0000 7f4cfdff9700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4cdc064ee0 0x7f4cdc067390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4cec001940 tx=0x7f4cec009040 comp rx=0 tx=0).stop 2023-03-03T06:25:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.007+0000 7f4cfdff9700 1 -- 172.21.15.3:0/3900840782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cf815c040 msgr2=0x7f4cf81592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:55.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.007+0000 7f4cfdff9700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cf815c040 0x7f4cf81592c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000eba0 tx=0x7f4cf000eeb0 comp rx=0 tx=0).stop 2023-03-03T06:25:55.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.011+0000 7f4cfdff9700 1 -- 172.21.15.3:0/3900840782 shutdown_connections 2023-03-03T06:25:55.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.011+0000 7f4cfdff9700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4cdc064ee0 0x7f4cdc067390 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:55.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.011+0000 7f4cfdff9700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cf815c040 0x7f4cf81592c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:55.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.011+0000 7f4cfdff9700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4cf80aee70 0x7f4cf815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:55.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.011+0000 7f4cfdff9700 1 --2- 172.21.15.3:0/3900840782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4cf806a780 0x7f4cf81650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:55.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.011+0000 7f4cfdff9700 1 -- 172.21.15.3:0/3900840782 >> 172.21.15.3:0/3900840782 conn(0x7f4cf8070a30 msgr2=0x7f4cf806c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:55.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.011+0000 7f4cfdff9700 1 -- 172.21.15.3:0/3900840782 shutdown_connections 2023-03-03T06:25:55.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:55.011+0000 7f4cfdff9700 1 -- 172.21.15.3:0/3900840782 wait complete. 2023-03-03T06:25:55.049 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-03-03T06:25:55.049 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-03-03T06:25:55.049 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-03-03T06:25:55.050 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-03-03T06:25:55.050 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-03-03T06:25:55.050 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-03-03T06:25:55.050 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-03-03T06:25:55.050 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-03-03T06:25:55.051 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-03-03T06:25:55.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:55.051 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-03-03T06:25:55.074 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-03T06:25:55.074 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-03T06:25:55.074 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-03-03T06:25:55.074 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:55.075 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-03-03T06:25:55.075 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:55.075 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:25:55.075 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:55.075 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-03-03T06:25:55.075 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-03-03T06:25:55.075 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:55.075 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-03-03T06:25:55.076 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-03-03T06:25:55.076 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42337sec preferred_lft 42337sec 2023-03-03T06:25:55.076 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2023-03-03T06:25:55.076 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:55.076 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:25:55.076 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:55.076 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-03-03T06:25:55.077 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:25:55.077 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:55.077 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-03-03T06:25:55.077 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-03-03T06:25:55.077 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:69:76:c1:31 brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:55.077 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-03-03T06:25:55.077 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:55.078 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-03-03T06:25:55.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:55.079 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:55.079 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:55.079 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2023-03-03T06:25:55.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2023-03-03T06:25:55.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2023-03-03T06:25:55.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-03-03T06:25:55.080 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:55.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:55.274 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-03-03T06:25:55.322 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-03-03T06:25:55.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:55.329 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-03-03T06:25:55.394 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-03-03T06:25:55.394 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-03-03T06:25:55.394 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-03-03T06:25:55.395 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-03-03T06:25:55.395 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-03-03T06:25:55.395 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-03-03T06:25:55.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:55.396 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:55.396 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:55.397 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-03-03T06:25:55.397 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-03-03T06:25:55.397 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-03-03T06:25:55.397 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:55.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:55.563 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-03-03T06:25:55.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:55.587 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2023-03-03T06:25:55.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:55.653 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:55.653 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:55.653 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-03-03T06:25:55.654 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-03-03T06:25:55.654 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:55.802 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-03-03T06:25:55.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:55.803 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:55.803 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:55.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-03-03T06:25:55.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-03-03T06:25:55.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-03-03T06:25:55.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-03-03T06:25:55.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-03-03T06:25:55.804 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:55.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:55.998 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:55.998 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:55.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-03-03T06:25:55.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2023-03-03T06:25:55.999 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:56.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-03T06:25:56.096 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-03-03T06:25:56.115 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-03-03T06:25:56.115 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-03T06:25:56.116 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-03-03T06:25:56.177 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-03-03T06:25:56.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:56.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-03-03T06:25:56.330 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-03-03T06:25:56.331 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-03-03T06:25:56.331 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-03-03T06:25:56.476 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-03-03T06:25:56.477 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-03-03T06:25:56.477 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-03-03T06:25:56.477 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.32:3300,172.21.15.32:3301" 2023-03-03T06:25:56.477 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2023-03-03T06:25:56.504 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-03-03T06:25:56.505 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-03-03T06:25:56.505 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-03-03T06:25:56.505 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.32:3300,172.21.15.32:3301" 2023-03-03T06:25:56.505 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2023-03-03T06:25:56.505 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.32:6789,172.21.15.32:6790" 2023-03-03T06:25:56.506 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.32:6789,172.21.15.32:6790:/ 2023-03-03T06:25:56.506 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=7d496254-7268-405f-a1d8-c8033dc8659a" will pass to kernel 2023-03-03T06:25:56.518 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-03-03T06:25:56.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:56.519 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-03-03T06:25:56.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:56.561 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-03-03T06:25:56.640 INFO:teuthology.orchestra.run.smithi003.stdout:client4828 2023-03-03T06:25:56.641 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:25:56.641 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:25:56.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:56.707 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-03-03T06:25:56.782 INFO:teuthology.orchestra.run.smithi003.stdout:client4828 2023-03-03T06:25:56.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:56.783 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-03-03T06:25:56.864 INFO:teuthology.orchestra.run.smithi003.stdout:client4828 2023-03-03T06:25:56.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:56.866 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/7d496254-7268-405f-a1d8-c8033dc8659a.client4828/status') 2023-03-03T06:25:56.951 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.4828 (3)172.21.15.3:0/3918952993 2023-03-03T06:25:56.951 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2023-03-03T06:25:56.952 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2023-03-03T06:25:56.952 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2023-03-03T06:25:56.952 INFO:teuthology.orchestra.run.smithi003.stderr:69 bytes copied, 0.000132679 s, 520 kB/s 2023-03-03T06:25:56.956 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-03-03T06:25:56.957 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-03-03T06:25:56.957 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-03-03T06:25:56.957 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-03-03T06:25:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.199+0000 7f3e50c76700 1 Processor -- start 2023-03-03T06:25:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.199+0000 7f3e50c76700 1 -- start start 2023-03-03T06:25:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.203+0000 7f3e50c76700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3e4c0ca9b0 0x7f3e4c15d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.203+0000 7f3e50c76700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4c15dc60 con 0x7f3e4c159440 2023-03-03T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.203+0000 7f3e50c76700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4c0ae720 con 0x7f3e4c159070 2023-03-03T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.203+0000 7f3e50c76700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4c0cae40 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.203+0000 7f3e42ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3e4c159440 0x7f3e4c0ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46956/0 (socket says 172.21.15.3:46956) 2023-03-03T06:25:57.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.203+0000 7f3e42ffd700 1 -- 172.21.15.3:0/3287143363 learned_addr learned my addr 172.21.15.3:0/3287143363 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:57.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.203+0000 7f3e4bfff700 1 --2- 172.21.15.3:0/3287143363 >> v2:172.21.15.32:3300/0 conn(0x7f3e4c0ca9b0 0x7f3e4c15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:57.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.203+0000 7f3e4affd700 1 -- 172.21.15.3:0/3287143363 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976438265 0 0) 0x7f3e34001110 con 0x7f3e4c159440 2023-03-03T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.203+0000 7f3e4affd700 1 -- 172.21.15.3:0/3287143363 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e380035d0 con 0x7f3e4c159440 2023-03-03T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.203+0000 7f3e4bfff700 1 -- 172.21.15.3:0/3287143363 >> v1:172.21.15.32:6789/0 conn(0x7f3e4c159070 legacy=0x7f3e4c0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.203+0000 7f3e4bfff700 1 -- 172.21.15.3:0/3287143363 >> v1:172.21.15.3:6789/0 conn(0x7f3e4c159440 legacy=0x7f3e4c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.203+0000 7f3e4bfff700 1 -- 172.21.15.3:0/3287143363 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c0ade80 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.207+0000 7f3e4bfff700 1 --2- 172.21.15.3:0/3287143363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e4c0ca9b0 0x7f3e4c15d720 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400bfa0 tx=0x7f3e4400d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d906e145416ac2e server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.207+0000 7f3e4affd700 1 -- 172.21.15.3:0/3287143363 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e44010070 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.207+0000 7f3e4affd700 1 -- 172.21.15.3:0/3287143363 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4400b2e0 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.207+0000 7f3e4affd700 1 -- 172.21.15.3:0/3287143363 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400dc40 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.207+0000 7f3e50c76700 1 -- 172.21.15.3:0/3287143363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e4c0ca9b0 msgr2=0x7f3e4c15d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:57.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.207+0000 7f3e50c76700 1 --2- 172.21.15.3:0/3287143363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e4c0ca9b0 0x7f3e4c15d720 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400bfa0 tx=0x7f3e4400d330 comp rx=0 tx=0).stop 2023-03-03T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.207+0000 7f3e50c76700 1 -- 172.21.15.3:0/3287143363 shutdown_connections 2023-03-03T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.207+0000 7f3e50c76700 1 --2- 172.21.15.3:0/3287143363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e4c0ca9b0 0x7f3e4c15d720 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.207+0000 7f3e50c76700 1 -- 172.21.15.3:0/3287143363 >> 172.21.15.3:0/3287143363 conn(0x7f3e4c070a30 msgr2=0x7f3e4c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.207+0000 7f3e50c76700 1 -- 172.21.15.3:0/3287143363 shutdown_connections 2023-03-03T06:25:57.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.207+0000 7f3e50c76700 1 -- 172.21.15.3:0/3287143363 wait complete. 2023-03-03T06:25:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.211+0000 7f3e50c76700 1 Processor -- start 2023-03-03T06:25:57.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.211+0000 7f3e50c76700 1 -- start start 2023-03-03T06:25:57.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.211+0000 7f3e50c76700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e4c0ca9b0 0x7f3e4c15c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.211+0000 7f3e50c76700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e4c159070 0x7f3e4c156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.211+0000 7f3e50c76700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3e4c159440 0x7f3e4c152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.211+0000 7f3e50c76700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c15c6c0 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.211+0000 7f3e50c76700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c157100 con 0x7f3e4c159070 2023-03-03T06:25:57.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.211+0000 7f3e50c76700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1573e0 con 0x7f3e4c159440 2023-03-03T06:25:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.211+0000 7f3e4b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e4c0ca9b0 0x7f3e4c15c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.211+0000 7f3e4b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e4c0ca9b0 0x7f3e4c15c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47994/0 (socket says 172.21.15.3:47994) 2023-03-03T06:25:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.211+0000 7f3e4b7fe700 1 -- 172.21.15.3:0/1313492932 learned_addr learned my addr 172.21.15.3:0/1313492932 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e4bfff700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3e4c159440 0x7f3e4c152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e42ffd700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e4c159070 0x7f3e4c156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e4b7fe700 1 -- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3e4c159440 msgr2=0x7f3e4c152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e4b7fe700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3e4c159440 0x7f3e4c152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e4b7fe700 1 -- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e4c159070 msgr2=0x7f3e4c156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e4b7fe700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e4c159070 0x7f3e4c156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e4b7fe700 1 -- 172.21.15.3:0/1313492932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4400bc80 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e4bfff700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3e4c159440 0x7f3e4c152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:25:57.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e4b7fe700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e4c0ca9b0 0x7f3e4c15c180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00c500 tx=0x7f3e3c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:57.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e48ff9700 1 -- 172.21.15.3:0/1313492932 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0052d0 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e48ff9700 1 -- 172.21.15.3:0/1313492932 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3c005430 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e50c76700 1 -- 172.21.15.3:0/1313492932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4c0ae440 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e48ff9700 1 -- 172.21.15.3:0/1313492932 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0055a0 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e50c76700 1 -- 172.21.15.3:0/1313492932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4c0ae160 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.215+0000 7f3e50c76700 1 -- 172.21.15.3:0/1313492932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4c0adbd0 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.219+0000 7f3e48ff9700 1 -- 172.21.15.3:0/1313492932 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3e3c019030 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.223+0000 7f3e48ff9700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3e2c064f10 0x7f3e2c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:57.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.223+0000 7f3e48ff9700 1 -- 172.21.15.3:0/1313492932 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3e3c081630 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.223+0000 7f3e42ffd700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3e2c064f10 0x7f3e2c0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:57.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.223+0000 7f3e42ffd700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3e2c064f10 0x7f3e2c0673c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0c5750 tx=0x7f3e34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:57.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.227+0000 7f3e48ff9700 1 -- 172.21.15.3:0/1313492932 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f3e3c04f3b0 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.274 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:25:57.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:25:57.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.443+0000 7f3e50c76700 1 -- 172.21.15.3:0/1313492932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3e4c073e90 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.443+0000 7f3e48ff9700 1 -- 172.21.15.3:0/1313492932 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3e3c0529d0 con 0x7f3e4c0ca9b0 2023-03-03T06:25:57.448 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-03-03T06:25:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.447+0000 7f3e50c76700 1 -- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3e2c064f10 msgr2=0x7f3e2c0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.447+0000 7f3e50c76700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3e2c064f10 0x7f3e2c0673c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0c5750 tx=0x7f3e34009040 comp rx=0 tx=0).stop 2023-03-03T06:25:57.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.447+0000 7f3e50c76700 1 -- 172.21.15.3:0/1313492932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e4c0ca9b0 msgr2=0x7f3e4c15c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.447+0000 7f3e50c76700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e4c0ca9b0 0x7f3e4c15c180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00c500 tx=0x7f3e3c00c8c0 comp rx=0 tx=0).stop 2023-03-03T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.447+0000 7f3e50c76700 1 -- 172.21.15.3:0/1313492932 shutdown_connections 2023-03-03T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.447+0000 7f3e50c76700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3e2c064f10 0x7f3e2c0673c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.447+0000 7f3e50c76700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3e4c159440 0x7f3e4c152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:57.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.447+0000 7f3e50c76700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e4c159070 0x7f3e4c156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:57.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.447+0000 7f3e50c76700 1 --2- 172.21.15.3:0/1313492932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e4c0ca9b0 0x7f3e4c15c180 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:57.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.447+0000 7f3e50c76700 1 -- 172.21.15.3:0/1313492932 >> 172.21.15.3:0/1313492932 conn(0x7f3e4c070a30 msgr2=0x7f3e4c0cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:57.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.447+0000 7f3e50c76700 1 -- 172.21.15.3:0/1313492932 shutdown_connections 2023-03-03T06:25:57.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:57.447+0000 7f3e50c76700 1 -- 172.21.15.3:0/1313492932 wait complete. 2023-03-03T06:25:57.487 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-03-03T06:25:57.488 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-03-03T06:25:57.488 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-03-03T06:25:57.488 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-03-03T06:25:57.488 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-03-03T06:25:57.489 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-03-03T06:25:57.489 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-03-03T06:25:57.489 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-03-03T06:25:57.489 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-03-03T06:25:57.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:57.490 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-03-03T06:25:57.512 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-03T06:25:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-03T06:25:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-03-03T06:25:57.513 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-03-03T06:25:57.514 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:57.514 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-03-03T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-03-03T06:25:57.515 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-03-03T06:25:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-03-03T06:25:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42334sec preferred_lft 42334sec 2023-03-03T06:25:57.516 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2023-03-03T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-03-03T06:25:57.517 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:25:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:57.518 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-03-03T06:25:57.518 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-03-03T06:25:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:69:76:c1:31 brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-03-03T06:25:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:57.519 INFO:teuthology.orchestra.run.smithi003.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-03T06:25:57.519 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 8e:e5:8f:ab:8a:c0 brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-03-03T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f8ed:b6ff:fede:6b5f/64 scope link 2023-03-03T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-03-03T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 8e:e5:8f:ab:8a:c0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-03-03T06:25:57.520 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::8ce5:8fff:feab:8ac0/64 scope link 2023-03-03T06:25:57.521 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:57.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:57.522 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-03-03T06:25:57.611 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-03-03T06:25:57.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:57.617 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-03-03T06:25:57.686 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-03-03T06:25:57.686 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-03-03T06:25:57.686 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-03-03T06:25:57.686 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-03-03T06:25:57.686 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-03-03T06:25:57.687 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-03-03T06:25:57.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:57.688 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:57.688 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:57.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-03-03T06:25:57.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-03-03T06:25:57.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-03-03T06:25:57.689 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:57.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:57.856 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-03-03T06:25:57.877 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-03-03T06:25:57.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:57.879 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2023-03-03T06:25:57.941 INFO:teuthology.orchestra.run.smithi003.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-03-03T06:25:57.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:57.943 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:57.943 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:57.944 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-03-03T06:25:57.944 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-03-03T06:25:57.944 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:58.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:58.083 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-03-03T06:25:58.130 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-03T06:25:58.130 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-03T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-03-03T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-03-03T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:58.131 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-03T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 4a:c1:e8:9e:7b:b0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-03-03T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-03-03T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::48c1:e8ff:fe9e:7bb0/64 scope link 2023-03-03T06:25:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:58.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:58.137 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-03-03T06:25:58.223 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-03T06:25:58.223 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-03T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-03-03T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-03-03T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-03T06:25:58.224 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 4a:c1:e8:9e:7b:b0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-03-03T06:25:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-03-03T06:25:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::48c1:e8ff:fe9e:7bb0/64 scope link 2023-03-03T06:25:58.225 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:58.230 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-03-03T06:25:58.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:58.231 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:58.231 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:58.231 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-03-03T06:25:58.232 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-03-03T06:25:58.232 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-03-03T06:25:58.232 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-03-03T06:25:58.232 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-03-03T06:25:58.232 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:58.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:58.484 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:25:58.485 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:25:58.485 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.1 up 2023-03-03T06:25:58.485 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.1 master ceph-brx 2023-03-03T06:25:58.485 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:25:58.586 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-03T06:25:58.587 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-03-03T06:25:58.608 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-03-03T06:25:58.609 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-03T06:25:58.609 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-03-03T06:25:58.677 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-03-03T06:25:58.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:58.678 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-03-03T06:25:58.828 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-03-03T06:25:58.828 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-03-03T06:25:58.829 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-03-03T06:25:58.863 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-03-03T06:25:58.864 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-03-03T06:25:58.864 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-03-03T06:25:58.864 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.32:3300,172.21.15.32:3301" 2023-03-03T06:25:58.864 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2023-03-03T06:25:58.887 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-03-03T06:25:58.888 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-03-03T06:25:58.888 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-03-03T06:25:58.888 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.32:3300,172.21.15.32:3301" 2023-03-03T06:25:58.888 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2023-03-03T06:25:58.888 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.32:6789,172.21.15.32:6790" 2023-03-03T06:25:58.888 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.32:6789,172.21.15.32:6790:/ 2023-03-03T06:25:58.889 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=7d496254-7268-405f-a1d8-c8033dc8659a" will pass to kernel 2023-03-03T06:25:58.901 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-03-03T06:25:58.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:58.902 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-03-03T06:25:58.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:58.944 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-03-03T06:25:59.018 INFO:teuthology.orchestra.run.smithi003.stdout:client4821 2023-03-03T06:25:59.019 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:25:59.020 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:25:59.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:59.084 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-03-03T06:25:59.160 INFO:teuthology.orchestra.run.smithi003.stdout:client4821 2023-03-03T06:25:59.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:59.161 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-03-03T06:25:59.236 INFO:teuthology.orchestra.run.smithi003.stdout:client4821 2023-03-03T06:25:59.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:59.237 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/7d496254-7268-405f-a1d8-c8033dc8659a.client4821/status') 2023-03-03T06:25:59.330 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.4821 (3)192.168.0.2:0/1251187688 2023-03-03T06:25:59.330 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2023-03-03T06:25:59.331 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2023-03-03T06:25:59.331 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2023-03-03T06:25:59.331 INFO:teuthology.orchestra.run.smithi003.stderr:69 bytes copied, 0.00024476 s, 282 kB/s 2023-03-03T06:25:59.337 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-03-03T06:25:59.338 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-03-03T06:25:59.338 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-03-03T06:25:59.338 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-03-03T06:25:59.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.595+0000 7f434f59e700 1 Processor -- start 2023-03-03T06:25:59.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.595+0000 7f434f59e700 1 -- start start 2023-03-03T06:25:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.595+0000 7f434f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43500ca6f0 0x7f43500cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.595+0000 7f434f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43500afd40 0x7f43500b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:59.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.595+0000 7f434f59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43500cf0f0 con 0x7f43501590e0 2023-03-03T06:25:59.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.595+0000 7f434f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43500cb0b0 con 0x7f43500afd40 2023-03-03T06:25:59.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.595+0000 7f434f59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43500cc510 con 0x7f43500ca6f0 2023-03-03T06:25:59.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.595+0000 7f434e59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f43501590e0 0x7f43500ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46962/0 (socket says 172.21.15.3:46962) 2023-03-03T06:25:59.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.595+0000 7f434e59c700 1 -- 172.21.15.3:0/226587233 learned_addr learned my addr 172.21.15.3:0/226587233 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:25:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.595+0000 7f434ed9d700 1 --2- 172.21.15.3:0/226587233 >> v2:172.21.15.32:3300/0 conn(0x7f43500afd40 0x7f43500b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623676853 0 0) 0x7f43500cf0f0 con 0x7f43501590e0 2023-03-03T06:25:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f433c0035d0 con 0x7f43501590e0 2023-03-03T06:25:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f4347fff700 1 --2- 172.21.15.3:0/226587233 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43500ca6f0 0x7f43500cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2543366043 0 0) 0x7f433c0035d0 con 0x7f43501590e0 2023-03-03T06:25:59.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43500ca6f0 msgr2=0x7f43500cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 --2- 172.21.15.3:0/226587233 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43500ca6f0 0x7f43500cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 >> v2:172.21.15.32:3300/0 conn(0x7f43500afd40 msgr2=0x7f43500b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 --2- 172.21.15.3:0/226587233 >> v2:172.21.15.32:3300/0 conn(0x7f43500afd40 0x7f43500b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43500cc7f0 con 0x7f43501590e0 2023-03-03T06:25:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f4340001670 con 0x7f43501590e0 2023-03-03T06:25:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 >> v1:172.21.15.3:6789/0 conn(0x7f43501590e0 legacy=0x7f43500ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 --2- 172.21.15.3:0/226587233 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43500ca6f0 0x7f433c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:59.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 --2- 172.21.15.3:0/226587233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433c003fa0 0x7f433c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 --2- 172.21.15.3:0/226587233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f433c004db0 0x7f433c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43500cb0b0 con 0x7f433c004db0 2023-03-03T06:25:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43500cc510 con 0x7f433c003fa0 2023-03-03T06:25:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c00fe70 con 0x7f43500ca6f0 2023-03-03T06:25:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434ed9d700 1 --2- 172.21.15.3:0/226587233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f433c004db0 0x7f433c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434e59c700 1 --2- 172.21.15.3:0/226587233 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43500ca6f0 0x7f433c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f4347fff700 1 --2- 172.21.15.3:0/226587233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433c003fa0 0x7f433c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434ed9d700 1 -- 172.21.15.3:0/226587233 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43500ca6f0 msgr2=0x7f433c003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434ed9d700 1 --2- 172.21.15.3:0/226587233 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f43500ca6f0 0x7f433c003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434ed9d700 1 -- 172.21.15.3:0/226587233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433c003fa0 msgr2=0x7f433c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:25:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434ed9d700 1 --2- 172.21.15.3:0/226587233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433c003fa0 0x7f433c004870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434ed9d700 1 -- 172.21.15.3:0/226587233 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c0035d0 con 0x7f433c004db0 2023-03-03T06:25:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434ed9d700 1 --2- 172.21.15.3:0/226587233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f433c004db0 0x7f433c005220 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4348009fc0 tx=0x7f433c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5afdabfe1c7f0ff server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:59.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434800ec30 con 0x7f433c004db0 2023-03-03T06:25:59.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434800f040 con 0x7f433c004db0 2023-03-03T06:25:59.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434dd9b700 1 -- 172.21.15.3:0/226587233 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348014480 con 0x7f433c004db0 2023-03-03T06:25:59.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434f59e700 1 -- 172.21.15.3:0/226587233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f433c004db0 msgr2=0x7f433c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:59.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434f59e700 1 --2- 172.21.15.3:0/226587233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f433c004db0 0x7f433c005220 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4348009fc0 tx=0x7f433c0058a0 comp rx=0 tx=0).stop 2023-03-03T06:25:59.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434ed9d700 1 -- 172.21.15.3:0/226587233 reap_dead start 2023-03-03T06:25:59.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434f59e700 1 -- 172.21.15.3:0/226587233 shutdown_connections 2023-03-03T06:25:59.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434f59e700 1 -- 172.21.15.3:0/226587233 >> 172.21.15.3:0/226587233 conn(0x7f4350070a30 msgr2=0x7f4350070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:59.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434f59e700 1 -- 172.21.15.3:0/226587233 shutdown_connections 2023-03-03T06:25:59.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.599+0000 7f434f59e700 1 -- 172.21.15.3:0/226587233 wait complete. 2023-03-03T06:25:59.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.603+0000 7f434f59e700 1 Processor -- start 2023-03-03T06:25:59.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.603+0000 7f434f59e700 1 -- start start 2023-03-03T06:25:59.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.603+0000 7f434f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433c004db0 0x7f4350159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:59.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.603+0000 7f434f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4350159bd0 0x7f43501566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:59.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.603+0000 7f434f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4350156c10 0x7f43501ede10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:59.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.603+0000 7f434f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4350157ec0 con 0x7f4350159bd0 2023-03-03T06:25:59.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.603+0000 7f434f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4350157060 con 0x7f433c004db0 2023-03-03T06:25:59.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.603+0000 7f434f59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43501572b0 con 0x7f4350156c10 2023-03-03T06:25:59.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f434e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433c004db0 0x7f4350159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:59.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f434e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433c004db0 0x7f4350159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:44930/0 (socket says 172.21.15.3:44930) 2023-03-03T06:25:59.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f434e59c700 1 -- 172.21.15.3:0/1422953019 learned_addr learned my addr 172.21.15.3:0/1422953019 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:25:59.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f4347fff700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4350159bd0 0x7f43501566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:59.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f434ed9d700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4350156c10 0x7f43501ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:59.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f434e59c700 1 -- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4350156c10 msgr2=0x7f43501ede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:59.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f434e59c700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4350156c10 0x7f43501ede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f434e59c700 1 -- 172.21.15.3:0/1422953019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4350159bd0 msgr2=0x7f43501566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f434e59c700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4350159bd0 0x7f43501566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f434e59c700 1 -- 172.21.15.3:0/1422953019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43500ccdb0 con 0x7f433c004db0 2023-03-03T06:25:59.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f434e59c700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433c004db0 0x7f4350159690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f434000eaf0 tx=0x7f434000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:59.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f4346ffd700 1 -- 172.21.15.3:0/1422953019 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340005e00 con 0x7f433c004db0 2023-03-03T06:25:59.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f4346ffd700 1 -- 172.21.15.3:0/1422953019 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434000c780 con 0x7f433c004db0 2023-03-03T06:25:59.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f4346ffd700 1 -- 172.21.15.3:0/1422953019 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434000cb30 con 0x7f433c004db0 2023-03-03T06:25:59.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f434f59e700 1 -- 172.21.15.3:0/1422953019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43500ccad0 con 0x7f433c004db0 2023-03-03T06:25:59.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.607+0000 7f434f59e700 1 -- 172.21.15.3:0/1422953019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43500cd090 con 0x7f433c004db0 2023-03-03T06:25:59.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.611+0000 7f434f59e700 1 -- 172.21.15.3:0/1422953019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4350073e90 con 0x7f433c004db0 2023-03-03T06:25:59.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.611+0000 7f4346ffd700 1 -- 172.21.15.3:0/1422953019 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f434000cc90 con 0x7f433c004db0 2023-03-03T06:25:59.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.615+0000 7f4346ffd700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4330064ee0 0x7f4330067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:25:59.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.615+0000 7f4346ffd700 1 -- 172.21.15.3:0/1422953019 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4340082440 con 0x7f433c004db0 2023-03-03T06:25:59.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.615+0000 7f4347fff700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4330064ee0 0x7f4330067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:25:59.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.619+0000 7f4347fff700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4330064ee0 0x7f4330067390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f43500b3fb0 tx=0x7f4338008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:25:59.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.623+0000 7f4346ffd700 1 -- 172.21.15.3:0/1422953019 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4340050240 con 0x7f433c004db0 2023-03-03T06:25:59.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.843+0000 7f434f59e700 1 -- 172.21.15.3:0/1422953019 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4350071840 con 0x7f433c004db0 2023-03-03T06:25:59.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.843+0000 7f4346ffd700 1 -- 172.21.15.3:0/1422953019 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4340057e60 con 0x7f433c004db0 2023-03-03T06:25:59.849 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-03-03T06:25:59.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.851+0000 7f434f59e700 1 -- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4330064ee0 msgr2=0x7f4330067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:59.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.851+0000 7f434f59e700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4330064ee0 0x7f4330067390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f43500b3fb0 tx=0x7f4338008040 comp rx=0 tx=0).stop 2023-03-03T06:25:59.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.851+0000 7f434f59e700 1 -- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433c004db0 msgr2=0x7f4350159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:25:59.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.851+0000 7f434f59e700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433c004db0 0x7f4350159690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f434000eaf0 tx=0x7f434000ee00 comp rx=0 tx=0).stop 2023-03-03T06:25:59.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.851+0000 7f434f59e700 1 -- 172.21.15.3:0/1422953019 shutdown_connections 2023-03-03T06:25:59.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.851+0000 7f434f59e700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4330064ee0 0x7f4330067390 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:59.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.851+0000 7f434f59e700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4350156c10 0x7f43501ede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:59.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.851+0000 7f434f59e700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4350159bd0 0x7f43501566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:59.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.851+0000 7f434f59e700 1 --2- 172.21.15.3:0/1422953019 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f433c004db0 0x7f4350159690 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:25:59.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.851+0000 7f434f59e700 1 -- 172.21.15.3:0/1422953019 >> 172.21.15.3:0/1422953019 conn(0x7f4350070a30 msgr2=0x7f435015d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:25:59.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.851+0000 7f434f59e700 1 -- 172.21.15.3:0/1422953019 shutdown_connections 2023-03-03T06:25:59.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:25:59.851+0000 7f434f59e700 1 -- 172.21.15.3:0/1422953019 wait complete. 2023-03-03T06:25:59.889 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-03-03T06:25:59.889 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-03-03T06:25:59.889 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-03-03T06:25:59.890 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-03-03T06:25:59.890 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-03-03T06:25:59.890 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-03-03T06:25:59.890 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-03-03T06:25:59.890 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-03-03T06:25:59.891 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-03-03T06:25:59.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:59.891 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-03-03T06:25:59.914 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-03T06:25:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-03T06:25:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-03-03T06:25:59.914 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-03-03T06:25:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:59.915 INFO:teuthology.orchestra.run.smithi032.stdout:2: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-03-03T06:25:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-03-03T06:25:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-03-03T06:25:59.915 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42344sec preferred_lft 42344sec 2023-03-03T06:25:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-03-03T06:25:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:59.916 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:25:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-03-03T06:25:59.916 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:25:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:59.916 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-03-03T06:25:59.917 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:25:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-03-03T06:25:59.917 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-03-03T06:25:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:24:42:3e:42 brd ff:ff:ff:ff:ff:ff 2023-03-03T06:25:59.917 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-03-03T06:25:59.918 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:25:59.918 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-03-03T06:25:59.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:25:59.919 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-03-03T06:25:59.919 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-03-03T06:25:59.920 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2023-03-03T06:25:59.920 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2023-03-03T06:25:59.920 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2023-03-03T06:25:59.920 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-03-03T06:25:59.921 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-03-03T06:26:00.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:00.097 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-03-03T06:26:00.143 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-03-03T06:26:00.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:00.148 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-03-03T06:26:00.226 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-03-03T06:26:00.227 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-03-03T06:26:00.227 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-03-03T06:26:00.227 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-03-03T06:26:00.227 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-03-03T06:26:00.228 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-03-03T06:26:00.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:00.229 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-03-03T06:26:00.229 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-03-03T06:26:00.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-03-03T06:26:00.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-03-03T06:26:00.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-03-03T06:26:00.230 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-03-03T06:26:00.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:00.410 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-03-03T06:26:00.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:00.434 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2023-03-03T06:26:00.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:00.498 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-03-03T06:26:00.498 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-03-03T06:26:00.499 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-03-03T06:26:00.499 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-03-03T06:26:00.499 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-03-03T06:26:00.643 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-03-03T06:26:00.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:00.644 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-03-03T06:26:00.644 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-03-03T06:26:00.644 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-03-03T06:26:00.644 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-03-03T06:26:00.644 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-03-03T06:26:00.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-03-03T06:26:00.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-03-03T06:26:00.645 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-03-03T06:26:00.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:00.881 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-03-03T06:26:00.882 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-03-03T06:26:00.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-03-03T06:26:00.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2023-03-03T06:26:00.882 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-03-03T06:26:00.972 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-03T06:26:00.973 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-03-03T06:26:00.994 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-03-03T06:26:00.995 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-03T06:26:00.996 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-03-03T06:26:01.060 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-03-03T06:26:01.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:01.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-03-03T06:26:01.218 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-03-03T06:26:01.218 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-03-03T06:26:01.218 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2023-03-03T06:26:01.361 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-03-03T06:26:01.362 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-03-03T06:26:01.362 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2023-03-03T06:26:01.362 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.32:3300,172.21.15.32:3301" 2023-03-03T06:26:01.362 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2023-03-03T06:26:01.387 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-03-03T06:26:01.387 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-03-03T06:26:01.388 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2023-03-03T06:26:01.388 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.32:3300,172.21.15.32:3301" 2023-03-03T06:26:01.388 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2023-03-03T06:26:01.388 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.32:6789,172.21.15.32:6790" 2023-03-03T06:26:01.388 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.32:6789,172.21.15.32:6790:/ 2023-03-03T06:26:01.388 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=7d496254-7268-405f-a1d8-c8033dc8659a" will pass to kernel 2023-03-03T06:26:01.400 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-03-03T06:26:01.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:01.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-03-03T06:26:01.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:01.444 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-03-03T06:26:01.534 INFO:teuthology.orchestra.run.smithi032.stdout:client54563 2023-03-03T06:26:01.534 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:26:01.535 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:01.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:01.597 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-03-03T06:26:01.672 INFO:teuthology.orchestra.run.smithi032.stdout:client54563 2023-03-03T06:26:01.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:01.672 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-03-03T06:26:01.757 INFO:teuthology.orchestra.run.smithi032.stdout:client54563 2023-03-03T06:26:01.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:01.758 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/7d496254-7268-405f-a1d8-c8033dc8659a.client54563/status') 2023-03-03T06:26:01.844 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.54563 (3)192.168.0.1:0/1621796718 2023-03-03T06:26:01.844 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2023-03-03T06:26:01.845 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2023-03-03T06:26:01.845 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2023-03-03T06:26:01.845 INFO:teuthology.orchestra.run.smithi032.stderr:70 bytes copied, 0.000100268 s, 698 kB/s 2023-03-03T06:26:01.848 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-03-03T06:26:01.848 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-03-03T06:26:01.848 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-03-03T06:26:01.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-03-03T06:26:02.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.051+0000 7fee1e328700 1 Processor -- start 2023-03-03T06:26:02.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.051+0000 7fee1e328700 1 -- start start 2023-03-03T06:26:02.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.051+0000 7fee1e328700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee180aea70 0x7fee180ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:02.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.051+0000 7fee1e328700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee180adcf0 0x7fee180ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:02.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.051+0000 7fee1e328700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee180ce190 con 0x7fee18160880 2023-03-03T06:26:02.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.051+0000 7fee1e328700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee180ae190 con 0x7fee180aea70 2023-03-03T06:26:02.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.051+0000 7fee1e328700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee180c99d0 con 0x7fee180adcf0 2023-03-03T06:26:02.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.055+0000 7fee1d326700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fee18160880 0x7fee180c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43304/0 (socket says 172.21.15.3:43304) 2023-03-03T06:26:02.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.055+0000 7fee1d326700 1 -- 172.21.15.3:0/2697565318 learned_addr learned my addr 172.21.15.3:0/2697565318 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:02.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.055+0000 7fee1db27700 1 --2- 172.21.15.3:0/2697565318 >> v2:172.21.15.32:3300/0 conn(0x7fee180adcf0 0x7fee180ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:02.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.055+0000 7fee1cb25700 1 --2- 172.21.15.3:0/2697565318 >> v2:172.21.15.3:3300/0 conn(0x7fee180aea70 0x7fee180ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:02.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.055+0000 7fee1cb25700 1 -- 172.21.15.3:0/2697565318 >> v2:172.21.15.32:3300/0 conn(0x7fee180adcf0 msgr2=0x7fee180ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:02.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.055+0000 7fee1cb25700 1 --2- 172.21.15.3:0/2697565318 >> v2:172.21.15.32:3300/0 conn(0x7fee180adcf0 0x7fee180ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:02.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.055+0000 7fee1cb25700 1 -- 172.21.15.3:0/2697565318 >> v1:172.21.15.3:6789/0 conn(0x7fee18160880 legacy=0x7fee180c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:02.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.055+0000 7fee1cb25700 1 -- 172.21.15.3:0/2697565318 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee180cd580 con 0x7fee180aea70 2023-03-03T06:26:02.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.055+0000 7fee1cb25700 1 --2- 172.21.15.3:0/2697565318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180aea70 0x7fee180ad7b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee180b13b0 tx=0x7fee08009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fbb17fda9639aa server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:02.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.055+0000 7fee0ffff700 1 -- 172.21.15.3:0/2697565318 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08011830 con 0x7fee180aea70 2023-03-03T06:26:02.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.055+0000 7fee0ffff700 1 -- 172.21.15.3:0/2697565318 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee08011990 con 0x7fee180aea70 2023-03-03T06:26:02.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.055+0000 7fee0ffff700 1 -- 172.21.15.3:0/2697565318 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08011d20 con 0x7fee180aea70 2023-03-03T06:26:02.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.059+0000 7fee1e328700 1 -- 172.21.15.3:0/2697565318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180aea70 msgr2=0x7fee180ad7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:02.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.059+0000 7fee1e328700 1 --2- 172.21.15.3:0/2697565318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180aea70 0x7fee180ad7b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee180b13b0 tx=0x7fee08009b30 comp rx=0 tx=0).stop 2023-03-03T06:26:02.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.059+0000 7fee1e328700 1 -- 172.21.15.3:0/2697565318 shutdown_connections 2023-03-03T06:26:02.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.059+0000 7fee1e328700 1 --2- 172.21.15.3:0/2697565318 >> v2:172.21.15.32:3300/0 conn(0x7fee180adcf0 0x7fee180ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:02.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.059+0000 7fee1e328700 1 --2- 172.21.15.3:0/2697565318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180aea70 0x7fee180ad7b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:02.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.059+0000 7fee1e328700 1 -- 172.21.15.3:0/2697565318 >> 172.21.15.3:0/2697565318 conn(0x7fee18070a30 msgr2=0x7fee18070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:02.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.059+0000 7fee1e328700 1 -- 172.21.15.3:0/2697565318 shutdown_connections 2023-03-03T06:26:02.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.059+0000 7fee1e328700 1 -- 172.21.15.3:0/2697565318 wait complete. 2023-03-03T06:26:02.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.063+0000 7fee1e328700 1 Processor -- start 2023-03-03T06:26:02.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.063+0000 7fee1e328700 1 -- start start 2023-03-03T06:26:02.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.063+0000 7fee1e328700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee180adcf0 0x7fee18162460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:02.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.063+0000 7fee1e328700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180aea70 0x7fee181fa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:02.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.063+0000 7fee1e328700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee18160880 0x7fee1814f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:02.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.063+0000 7fee1e328700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee181507c0 con 0x7fee180aea70 2023-03-03T06:26:02.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.063+0000 7fee1e328700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1814d4d0 con 0x7fee18160880 2023-03-03T06:26:02.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.063+0000 7fee1e328700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee18150200 con 0x7fee180adcf0 2023-03-03T06:26:02.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.063+0000 7fee1cb25700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180aea70 0x7fee181fa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:02.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.063+0000 7fee1cb25700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180aea70 0x7fee181fa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59048/0 (socket says 172.21.15.3:59048) 2023-03-03T06:26:02.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.063+0000 7fee1cb25700 1 -- 172.21.15.3:0/4161562871 learned_addr learned my addr 172.21.15.3:0/4161562871 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:02.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.063+0000 7fee1db27700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee18160880 0x7fee1814f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:02.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.063+0000 7fee1d326700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee180adcf0 0x7fee18162460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:02.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.067+0000 7fee1cb25700 1 -- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee180adcf0 msgr2=0x7fee18162460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:02.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.067+0000 7fee1cb25700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee180adcf0 0x7fee18162460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:02.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.067+0000 7fee1cb25700 1 -- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee18160880 msgr2=0x7fee1814f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:02.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.067+0000 7fee1cb25700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee18160880 0x7fee1814f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:02.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.067+0000 7fee1cb25700 1 -- 172.21.15.3:0/4161562871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee080094b0 con 0x7fee180aea70 2023-03-03T06:26:02.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.067+0000 7fee1d326700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee180adcf0 0x7fee18162460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:02.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.067+0000 7fee1cb25700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180aea70 0x7fee181fa460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee08000c00 tx=0x7fee08004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:02.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.067+0000 7fee0dffb700 1 -- 172.21.15.3:0/4161562871 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee080038c0 con 0x7fee180aea70 2023-03-03T06:26:02.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.067+0000 7fee1e328700 1 -- 172.21.15.3:0/4161562871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee180cdb40 con 0x7fee180aea70 2023-03-03T06:26:02.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.067+0000 7fee0dffb700 1 -- 172.21.15.3:0/4161562871 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0801e070 con 0x7fee180aea70 2023-03-03T06:26:02.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.067+0000 7fee0dffb700 1 -- 172.21.15.3:0/4161562871 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08003c20 con 0x7fee180aea70 2023-03-03T06:26:02.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.067+0000 7fee1e328700 1 -- 172.21.15.3:0/4161562871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee180cd860 con 0x7fee180aea70 2023-03-03T06:26:02.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.067+0000 7fee1e328700 1 -- 172.21.15.3:0/4161562871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee180cde20 con 0x7fee180aea70 2023-03-03T06:26:02.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.071+0000 7fee0dffb700 1 -- 172.21.15.3:0/4161562871 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fee0800f050 con 0x7fee180aea70 2023-03-03T06:26:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.071+0000 7fee0dffb700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fedfc064ee0 0x7fedfc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:02.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.071+0000 7fee0dffb700 1 -- 172.21.15.3:0/4161562871 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fee08083240 con 0x7fee180aea70 2023-03-03T06:26:02.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.071+0000 7fee1d326700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fedfc064ee0 0x7fedfc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:02.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.075+0000 7fee1d326700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fedfc064ee0 0x7fedfc067390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fee14005930 tx=0x7fee14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:02.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.079+0000 7fee0dffb700 1 -- 172.21.15.3:0/4161562871 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fee08050fc0 con 0x7fee180aea70 2023-03-03T06:26:02.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.295+0000 7fee1e328700 1 -- 172.21.15.3:0/4161562871 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fee18073e90 con 0x7fee180aea70 2023-03-03T06:26:02.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.299+0000 7fee0dffb700 1 -- 172.21.15.3:0/4161562871 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fee080545e0 con 0x7fee180aea70 2023-03-03T06:26:02.302 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-03-03T06:26:02.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.303+0000 7fee1e328700 1 -- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fedfc064ee0 msgr2=0x7fedfc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:02.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.303+0000 7fee1e328700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fedfc064ee0 0x7fedfc067390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fee14005930 tx=0x7fee14009040 comp rx=0 tx=0).stop 2023-03-03T06:26:02.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.303+0000 7fee1e328700 1 -- 172.21.15.3:0/4161562871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180aea70 msgr2=0x7fee181fa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:02.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.303+0000 7fee1e328700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180aea70 0x7fee181fa460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee08000c00 tx=0x7fee08004770 comp rx=0 tx=0).stop 2023-03-03T06:26:02.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.303+0000 7fee1e328700 1 -- 172.21.15.3:0/4161562871 shutdown_connections 2023-03-03T06:26:02.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.303+0000 7fee1e328700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fedfc064ee0 0x7fedfc067390 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:02.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.303+0000 7fee1e328700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee18160880 0x7fee1814f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:02.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.303+0000 7fee1e328700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee180aea70 0x7fee181fa460 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:02.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.303+0000 7fee1e328700 1 --2- 172.21.15.3:0/4161562871 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fee180adcf0 0x7fee18162460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:02.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.303+0000 7fee1e328700 1 -- 172.21.15.3:0/4161562871 >> 172.21.15.3:0/4161562871 conn(0x7fee18070a30 msgr2=0x7fee1815ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:02.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.303+0000 7fee1e328700 1 -- 172.21.15.3:0/4161562871 shutdown_connections 2023-03-03T06:26:02.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:02.303+0000 7fee1e328700 1 -- 172.21.15.3:0/4161562871 wait complete. 2023-03-03T06:26:02.340 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-03-03T06:26:02.341 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-03-03T06:26:02.341 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-03-03T06:26:02.341 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-03-03T06:26:02.341 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-03-03T06:26:02.342 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-03-03T06:26:02.342 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-03-03T06:26:02.342 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-03-03T06:26:02.342 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-03-03T06:26:02.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:02.343 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-03-03T06:26:02.365 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-03T06:26:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-03T06:26:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-03-03T06:26:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-03-03T06:26:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:02.366 INFO:teuthology.orchestra.run.smithi032.stdout:2: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-03-03T06:26:02.366 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-03-03T06:26:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-03-03T06:26:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-03-03T06:26:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42341sec preferred_lft 42341sec 2023-03-03T06:26:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-03-03T06:26:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:02.367 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:26:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-03-03T06:26:02.367 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-03-03T06:26:02.368 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:26:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-03-03T06:26:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-03-03T06:26:02.368 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:26:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-03-03T06:26:02.368 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-03-03T06:26:02.368 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-03-03T06:26:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:24:42:3e:42 brd ff:ff:ff:ff:ff:ff 2023-03-03T06:26:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-03-03T06:26:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:02.369 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-03T06:26:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 26:95:df:5b:a0:45 brd ff:ff:ff:ff:ff:ff 2023-03-03T06:26:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-03-03T06:26:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:02.369 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::7b:ddff:fecd:98f0/64 scope link 2023-03-03T06:26:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:02.370 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-03-03T06:26:02.370 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 26:95:df:5b:a0:45 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-03-03T06:26:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::2495:dfff:fe5b:a045/64 scope link tentative 2023-03-03T06:26:02.371 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:02.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:02.372 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-03-03T06:26:02.464 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-03-03T06:26:02.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:02.470 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-03-03T06:26:02.535 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-03-03T06:26:02.535 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-03-03T06:26:02.536 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-03-03T06:26:02.536 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-03-03T06:26:02.536 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-03-03T06:26:02.536 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-03-03T06:26:02.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:02.538 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-03-03T06:26:02.539 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-03-03T06:26:02.539 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-03-03T06:26:02.539 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-03-03T06:26:02.539 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-03-03T06:26:02.539 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-03-03T06:26:02.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:02.701 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-03-03T06:26:02.722 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-03-03T06:26:02.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:02.723 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2023-03-03T06:26:02.792 INFO:teuthology.orchestra.run.smithi032.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-03-03T06:26:02.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:02.794 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-03-03T06:26:02.794 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-03-03T06:26:02.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-03-03T06:26:02.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-03-03T06:26:02.795 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-03-03T06:26:02.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:02.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-03-03T06:26:02.979 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-03T06:26:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-03T06:26:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-03-03T06:26:02.979 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-03-03T06:26:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:02.980 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-03T06:26:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 96:13:9a:17:bb:73 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-03-03T06:26:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-03-03T06:26:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:02.980 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::9413:9aff:fe17:bb73/64 scope link 2023-03-03T06:26:02.981 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:02.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:02.985 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-03-03T06:26:03.070 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-03T06:26:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-03T06:26:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-03-03T06:26:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-03-03T06:26:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:03.071 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-03T06:26:03.071 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 96:13:9a:17:bb:73 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-03-03T06:26:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-03-03T06:26:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::9413:9aff:fe17:bb73/64 scope link 2023-03-03T06:26:03.072 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:26:03.077 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-03-03T06:26:03.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:03.080 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-03-03T06:26:03.081 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-03-03T06:26:03.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-03-03T06:26:03.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-03-03T06:26:03.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-03-03T06:26:03.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-03-03T06:26:03.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-03-03T06:26:03.083 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-03-03T06:26:03.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:03.329 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-03-03T06:26:03.330 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-03-03T06:26:03.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.1 up 2023-03-03T06:26:03.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.1 master ceph-brx 2023-03-03T06:26:03.330 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-03-03T06:26:03.424 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-03T06:26:03.425 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-03-03T06:26:03.444 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-03-03T06:26:03.445 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-03T06:26:03.445 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-03-03T06:26:03.507 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-03-03T06:26:03.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:03.508 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-03-03T06:26:03.660 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-03-03T06:26:03.661 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-03-03T06:26:03.661 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2023-03-03T06:26:03.695 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-03-03T06:26:03.695 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-03-03T06:26:03.695 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2023-03-03T06:26:03.695 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.32:3300,172.21.15.32:3301" 2023-03-03T06:26:03.696 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2023-03-03T06:26:03.717 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-03-03T06:26:03.718 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-03-03T06:26:03.718 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2023-03-03T06:26:03.718 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.32:3300,172.21.15.32:3301" 2023-03-03T06:26:03.718 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: switching to using v1 address with old syntax 2023-03-03T06:26:03.718 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.32:6789,172.21.15.32:6790" 2023-03-03T06:26:03.718 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.32:6789,172.21.15.32:6790:/ 2023-03-03T06:26:03.719 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=7d496254-7268-405f-a1d8-c8033dc8659a" will pass to kernel 2023-03-03T06:26:03.732 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-03-03T06:26:03.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:03.732 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-03-03T06:26:03.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:03.772 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-03-03T06:26:03.853 INFO:teuthology.orchestra.run.smithi032.stdout:client54566 2023-03-03T06:26:03.854 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:26:03.854 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:03.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:03.920 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-03-03T06:26:04.002 INFO:teuthology.orchestra.run.smithi032.stdout:client54566 2023-03-03T06:26:04.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:04.003 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-03-03T06:26:04.081 INFO:teuthology.orchestra.run.smithi032.stdout:client54566 2023-03-03T06:26:04.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:04.082 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/7d496254-7268-405f-a1d8-c8033dc8659a.client54566/status') 2023-03-03T06:26:04.175 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.54566 (3)192.168.0.2:0/389355129 2023-03-03T06:26:04.175 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2023-03-03T06:26:04.176 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2023-03-03T06:26:04.176 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2023-03-03T06:26:04.176 INFO:teuthology.orchestra.run.smithi032.stderr:69 bytes copied, 0.000156325 s, 441 kB/s 2023-03-03T06:26:04.181 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-03-03T06:26:04.206 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:04.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.411+0000 7ff79e46d700 1 Processor -- start 2023-03-03T06:26:04.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.411+0000 7ff79e46d700 1 -- start start 2023-03-03T06:26:04.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.415+0000 7ff79e46d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7980addf0 0x7ff7980b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.415+0000 7ff79e46d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7980ca150 con 0x7ff7980cacf0 2023-03-03T06:26:04.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.415+0000 7ff79d46b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7980addf0 0x7ff7980b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.415+0000 7ff79e46d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff79815e9c0 con 0x7ff7980b1b70 2023-03-03T06:26:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.415+0000 7ff79e46d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7980b1180 con 0x7ff7980addf0 2023-03-03T06:26:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.415+0000 7ff79d46b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7980addf0 0x7ff7980b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59054/0 (socket says 172.21.15.3:59054) 2023-03-03T06:26:04.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.415+0000 7ff79d46b700 1 -- 172.21.15.3:0/1585694394 learned_addr learned my addr 172.21.15.3:0/1585694394 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:04.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.415+0000 7ff79d46b700 1 -- 172.21.15.3:0/1585694394 >> v1:172.21.15.32:6789/0 conn(0x7ff7980b1b70 legacy=0x7ff7980ca5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:04.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.415+0000 7ff79d46b700 1 -- 172.21.15.3:0/1585694394 >> v1:172.21.15.3:6789/0 conn(0x7ff7980cacf0 legacy=0x7ff7980cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:04.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.419+0000 7ff79d46b700 1 -- 172.21.15.3:0/1585694394 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79815eca0 con 0x7ff7980addf0 2023-03-03T06:26:04.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.419+0000 7ff79d46b700 1 --2- 172.21.15.3:0/1585694394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7980addf0 0x7ff7980b1630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff794009910 tx=0x7ff794009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453e137ec7ecabb0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.419+0000 7ff78ffff700 1 -- 172.21.15.3:0/1585694394 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79400ba60 con 0x7ff7980addf0 2023-03-03T06:26:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.419+0000 7ff78ffff700 1 -- 172.21.15.3:0/1585694394 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79400bbc0 con 0x7ff7980addf0 2023-03-03T06:26:04.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.419+0000 7ff78ffff700 1 -- 172.21.15.3:0/1585694394 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794009e60 con 0x7ff7980addf0 2023-03-03T06:26:04.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.419+0000 7ff79e46d700 1 -- 172.21.15.3:0/1585694394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7980addf0 msgr2=0x7ff7980b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:04.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.419+0000 7ff79e46d700 1 --2- 172.21.15.3:0/1585694394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7980addf0 0x7ff7980b1630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff794009910 tx=0x7ff794009c20 comp rx=0 tx=0).stop 2023-03-03T06:26:04.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.419+0000 7ff79e46d700 1 -- 172.21.15.3:0/1585694394 shutdown_connections 2023-03-03T06:26:04.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.419+0000 7ff79e46d700 1 --2- 172.21.15.3:0/1585694394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7980addf0 0x7ff7980b1630 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:04.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.419+0000 7ff79e46d700 1 -- 172.21.15.3:0/1585694394 >> 172.21.15.3:0/1585694394 conn(0x7ff798070a30 msgr2=0x7ff798070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:04.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.419+0000 7ff79e46d700 1 -- 172.21.15.3:0/1585694394 shutdown_connections 2023-03-03T06:26:04.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.419+0000 7ff79e46d700 1 -- 172.21.15.3:0/1585694394 wait complete. 2023-03-03T06:26:04.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.423+0000 7ff79e46d700 1 Processor -- start 2023-03-03T06:26:04.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.423+0000 7ff79e46d700 1 -- start start 2023-03-03T06:26:04.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.423+0000 7ff79e46d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7980addf0 0x7ff798156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:04.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.423+0000 7ff79e46d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7980b1b70 0x7ff79814ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:04.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.423+0000 7ff79e46d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7980cacf0 0x7ff7980ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:04.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79e46d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79814b1d0 con 0x7ff7980b1b70 2023-03-03T06:26:04.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79e46d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980ccb80 con 0x7ff7980addf0 2023-03-03T06:26:04.430 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79e46d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980c72d0 con 0x7ff7980cacf0 2023-03-03T06:26:04.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79cc6a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7980b1b70 0x7ff79814ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:04.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79cc6a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7980b1b70 0x7ff79814ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59062/0 (socket says 172.21.15.3:59062) 2023-03-03T06:26:04.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79cc6a700 1 -- 172.21.15.3:0/2463481865 learned_addr learned my addr 172.21.15.3:0/2463481865 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:04.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79d46b700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7980addf0 0x7ff798156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:04.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79dc6c700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7980cacf0 0x7ff7980ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:04.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79cc6a700 1 -- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7980cacf0 msgr2=0x7ff7980ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:04.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79cc6a700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7980cacf0 0x7ff7980ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:04.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79cc6a700 1 -- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7980addf0 msgr2=0x7ff798156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:04.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79cc6a700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7980addf0 0x7ff798156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:04.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79cc6a700 1 -- 172.21.15.3:0/2463481865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff794009570 con 0x7ff7980b1b70 2023-03-03T06:26:04.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79dc6c700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7980cacf0 0x7ff7980ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:04.433 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79cc6a700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7980b1b70 0x7ff79814ac90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff78800c4d0 tx=0x7ff78800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:04.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff78dffb700 1 -- 172.21.15.3:0/2463481865 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7880052d0 con 0x7ff7980b1b70 2023-03-03T06:26:04.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff78dffb700 1 -- 172.21.15.3:0/2463481865 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff788005430 con 0x7ff7980b1b70 2023-03-03T06:26:04.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff78dffb700 1 -- 172.21.15.3:0/2463481865 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7880055a0 con 0x7ff7980b1b70 2023-03-03T06:26:04.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79e46d700 1 -- 172.21.15.3:0/2463481865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7980c9de0 con 0x7ff7980b1b70 2023-03-03T06:26:04.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.427+0000 7ff79e46d700 1 -- 172.21.15.3:0/2463481865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7980c9b00 con 0x7ff7980b1b70 2023-03-03T06:26:04.434 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.431+0000 7ff79e46d700 1 -- 172.21.15.3:0/2463481865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7980ae1c0 con 0x7ff7980b1b70 2023-03-03T06:26:04.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.431+0000 7ff78dffb700 1 -- 172.21.15.3:0/2463481865 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff788019030 con 0x7ff7980b1b70 2023-03-03T06:26:04.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.435+0000 7ff78dffb700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff77c064ee0 0x7ff77c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:04.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.435+0000 7ff78dffb700 1 -- 172.21.15.3:0/2463481865 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff7880815a0 con 0x7ff7980b1b70 2023-03-03T06:26:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.435+0000 7ff79d46b700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff77c064ee0 0x7ff77c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.435+0000 7ff79d46b700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff77c064ee0 0x7ff77c067390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff794006f80 tx=0x7ff794008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.439+0000 7ff78dffb700 1 -- 172.21.15.3:0/2463481865 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff78804f320 con 0x7ff7980b1b70 2023-03-03T06:26:04.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.619+0000 7ff79e46d700 1 -- 172.21.15.3:0/2463481865 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7980cc370 con 0x7ff7980b1b70 2023-03-03T06:26:04.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.623+0000 7ff78dffb700 1 -- 172.21.15.3:0/2463481865 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19652 (secure 0 0 0) 0x7ff788052940 con 0x7ff7980b1b70 2023-03-03T06:26:04.629 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:04.630 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:53.877945+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:04.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.627+0000 7ff79e46d700 1 -- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff77c064ee0 msgr2=0x7ff77c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:04.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.627+0000 7ff79e46d700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff77c064ee0 0x7ff77c067390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff794006f80 tx=0x7ff794008030 comp rx=0 tx=0).stop 2023-03-03T06:26:04.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.627+0000 7ff79e46d700 1 -- 172.21.15.3:0/2463481865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7980b1b70 msgr2=0x7ff79814ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:04.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.627+0000 7ff79e46d700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7980b1b70 0x7ff79814ac90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff78800c4d0 tx=0x7ff78800c7e0 comp rx=0 tx=0).stop 2023-03-03T06:26:04.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.631+0000 7ff79e46d700 1 -- 172.21.15.3:0/2463481865 shutdown_connections 2023-03-03T06:26:04.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.631+0000 7ff79e46d700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff77c064ee0 0x7ff77c067390 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:04.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.631+0000 7ff79e46d700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff7980cacf0 0x7ff7980ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:04.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.631+0000 7ff79e46d700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7980b1b70 0x7ff79814ac90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:04.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.631+0000 7ff79e46d700 1 --2- 172.21.15.3:0/2463481865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7980addf0 0x7ff798156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:04.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.631+0000 7ff79e46d700 1 -- 172.21.15.3:0/2463481865 >> 172.21.15.3:0/2463481865 conn(0x7ff798070a30 msgr2=0x7ff7980cb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:04.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.631+0000 7ff79e46d700 1 -- 172.21.15.3:0/2463481865 shutdown_connections 2023-03-03T06:26:04.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.631+0000 7ff79e46d700 1 -- 172.21.15.3:0/2463481865 wait complete. 2023-03-03T06:26:04.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-03T06:25:11.468534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-03T06:25:12.813283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-03T06:25:16.382800+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-03T06:26:04.670 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:26:04.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.875+0000 7f8ba6aa4700 1 Processor -- start 2023-03-03T06:26:04.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.875+0000 7f8ba6aa4700 1 -- start start 2023-03-03T06:26:04.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.879+0000 7f8ba6aa4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8ba00b0fd0 0x7f8ba0163cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:04.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.879+0000 7f8ba6aa4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba01595a0 con 0x7f8ba006c590 2023-03-03T06:26:04.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.879+0000 7f8ba6aa4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba006ca50 con 0x7f8ba01591d0 2023-03-03T06:26:04.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.879+0000 7f8ba6aa4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba006d6d0 con 0x7f8ba00b0fd0 2023-03-03T06:26:04.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.879+0000 7f8ba52a1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8ba006c590 0x7f8ba00b08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43320/0 (socket says 172.21.15.3:43320) 2023-03-03T06:26:04.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.879+0000 7f8ba52a1700 1 -- 172.21.15.3:0/1735953693 learned_addr learned my addr 172.21.15.3:0/1735953693 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:04.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.879+0000 7f8ba62a3700 1 --2- 172.21.15.3:0/1735953693 >> v2:172.21.15.32:3300/0 conn(0x7f8ba00b0fd0 0x7f8ba0163cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:04.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.879+0000 7f8ba4aa0700 1 -- 172.21.15.3:0/1735953693 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723238618 0 0) 0x7f8ba01595a0 con 0x7f8ba006c590 2023-03-03T06:26:04.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.879+0000 7f8ba4aa0700 1 -- 172.21.15.3:0/1735953693 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b8c0035d0 con 0x7f8ba006c590 2023-03-03T06:26:04.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.879+0000 7f8ba62a3700 1 -- 172.21.15.3:0/1735953693 >> v1:172.21.15.32:6789/0 conn(0x7f8ba01591d0 legacy=0x7f8ba00cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:04.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.879+0000 7f8ba62a3700 1 -- 172.21.15.3:0/1735953693 >> v1:172.21.15.3:6789/0 conn(0x7f8ba006c590 legacy=0x7f8ba00b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:04.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.879+0000 7f8ba4aa0700 1 -- 172.21.15.3:0/1735953693 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3278839244 0 0) 0x7f8b8c0035d0 con 0x7f8ba006c590 2023-03-03T06:26:04.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.879+0000 7f8ba62a3700 1 -- 172.21.15.3:0/1735953693 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba006cd30 con 0x7f8ba00b0fd0 2023-03-03T06:26:04.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.883+0000 7f8ba62a3700 1 --2- 172.21.15.3:0/1735953693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba00b0fd0 0x7f8ba0163cf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009d90 tx=0x7f8b9800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdc944c3da60fc85 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:04.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.883+0000 7f8ba4aa0700 1 -- 172.21.15.3:0/1735953693 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9800e070 con 0x7f8ba00b0fd0 2023-03-03T06:26:04.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.883+0000 7f8ba4aa0700 1 -- 172.21.15.3:0/1735953693 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b98003a40 con 0x7f8ba00b0fd0 2023-03-03T06:26:04.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.883+0000 7f8ba4aa0700 1 -- 172.21.15.3:0/1735953693 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90001670 con 0x7f8ba00b0fd0 2023-03-03T06:26:04.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.883+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/1735953693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba00b0fd0 msgr2=0x7f8ba0163cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:04.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.883+0000 7f8ba6aa4700 1 --2- 172.21.15.3:0/1735953693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba00b0fd0 0x7f8ba0163cf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009d90 tx=0x7f8b9800b330 comp rx=0 tx=0).stop 2023-03-03T06:26:04.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.883+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/1735953693 shutdown_connections 2023-03-03T06:26:04.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.883+0000 7f8ba6aa4700 1 --2- 172.21.15.3:0/1735953693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba00b0fd0 0x7f8ba0163cf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:04.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.883+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/1735953693 >> 172.21.15.3:0/1735953693 conn(0x7f8ba00607c0 msgr2=0x7f8ba0060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:04.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.883+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/1735953693 shutdown_connections 2023-03-03T06:26:04.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.883+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/1735953693 wait complete. 2023-03-03T06:26:04.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba6aa4700 1 Processor -- start 2023-03-03T06:26:04.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba6aa4700 1 -- start start 2023-03-03T06:26:04.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba6aa4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8ba006c590 0x7f8ba01e2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:04.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba6aa4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba00b0fd0 0x7f8ba014df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:04.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba6aa4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ba01591d0 0x7f8ba014e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:04.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba6aa4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba00c2b80 con 0x7f8ba01591d0 2023-03-03T06:26:04.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba6aa4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba014e980 con 0x7f8ba00b0fd0 2023-03-03T06:26:04.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba6aa4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba00c51a0 con 0x7f8ba006c590 2023-03-03T06:26:04.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba62a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ba01591d0 0x7f8ba014e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:04.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba62a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ba01591d0 0x7f8ba014e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59072/0 (socket says 172.21.15.3:59072) 2023-03-03T06:26:04.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba62a3700 1 -- 172.21.15.3:0/532805372 learned_addr learned my addr 172.21.15.3:0/532805372 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:04.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba52a1700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba00b0fd0 0x7f8ba014df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:04.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba5aa2700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8ba006c590 0x7f8ba01e2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:04.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba62a3700 1 -- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8ba006c590 msgr2=0x7f8ba01e2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:04.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba62a3700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8ba006c590 0x7f8ba01e2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:04.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba62a3700 1 -- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba00b0fd0 msgr2=0x7f8ba014df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:04.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba62a3700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba00b0fd0 0x7f8ba014df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:04.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba62a3700 1 -- 172.21.15.3:0/532805372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b98009a70 con 0x7f8ba01591d0 2023-03-03T06:26:04.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.887+0000 7f8ba52a1700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba00b0fd0 0x7f8ba014df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:04.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.891+0000 7f8ba62a3700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ba01591d0 0x7f8ba014e440 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8b98004000 tx=0x7f8b98004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:04.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.891+0000 7f8b967fc700 1 -- 172.21.15.3:0/532805372 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b98004720 con 0x7f8ba01591d0 2023-03-03T06:26:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.891+0000 7f8b967fc700 1 -- 172.21.15.3:0/532805372 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b98004880 con 0x7f8ba01591d0 2023-03-03T06:26:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.891+0000 7f8b967fc700 1 -- 172.21.15.3:0/532805372 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b98012670 con 0x7f8ba01591d0 2023-03-03T06:26:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.891+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/532805372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba006d2f0 con 0x7f8ba01591d0 2023-03-03T06:26:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.891+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/532805372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba006d010 con 0x7f8ba01591d0 2023-03-03T06:26:04.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.891+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/532805372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba00caa20 con 0x7f8ba01591d0 2023-03-03T06:26:04.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.895+0000 7f8b967fc700 1 -- 172.21.15.3:0/532805372 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8b9802c070 con 0x7f8ba01591d0 2023-03-03T06:26:04.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.895+0000 7f8b967fc700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8b84064ee0 0x7f8b84067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:04.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.895+0000 7f8b967fc700 1 -- 172.21.15.3:0/532805372 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8b98083ed0 con 0x7f8ba01591d0 2023-03-03T06:26:04.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.899+0000 7f8ba5aa2700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8b84064ee0 0x7f8b84067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:04.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.899+0000 7f8ba5aa2700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8b84064ee0 0x7f8b84067390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00ac3a0 tx=0x7f8b9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:04.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:04.903+0000 7f8b967fc700 1 -- 172.21.15.3:0/532805372 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f8b98051c50 con 0x7f8ba01591d0 2023-03-03T06:26:05.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.079+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/532805372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ba0073e90 con 0x7f8ba01591d0 2023-03-03T06:26:05.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.079+0000 7f8b967fc700 1 -- 172.21.15.3:0/532805372 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b98055270 con 0x7f8ba01591d0 2023-03-03T06:26:05.085 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:26:05.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.087+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8b84064ee0 msgr2=0x7f8b84067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:05.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.087+0000 7f8ba6aa4700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8b84064ee0 0x7f8b84067390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00ac3a0 tx=0x7f8b9c009040 comp rx=0 tx=0).stop 2023-03-03T06:26:05.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.087+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/532805372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ba01591d0 msgr2=0x7f8ba014e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:05.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.087+0000 7f8ba6aa4700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ba01591d0 0x7f8ba014e440 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8b98004000 tx=0x7f8b98004af0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.087+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/532805372 shutdown_connections 2023-03-03T06:26:05.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.087+0000 7f8ba6aa4700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8b84064ee0 0x7f8b84067390 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.087+0000 7f8ba6aa4700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ba01591d0 0x7f8ba014e440 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.087+0000 7f8ba6aa4700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba00b0fd0 0x7f8ba014df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.087+0000 7f8ba6aa4700 1 --2- 172.21.15.3:0/532805372 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8ba006c590 0x7f8ba01e2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.087+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/532805372 >> 172.21.15.3:0/532805372 conn(0x7f8ba00607c0 msgr2=0x7f8ba00af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:05.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.087+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/532805372 shutdown_connections 2023-03-03T06:26:05.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.087+0000 7f8ba6aa4700 1 -- 172.21.15.3:0/532805372 wait complete. 2023-03-03T06:26:05.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:26:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.327+0000 7f339f59e700 1 Processor -- start 2023-03-03T06:26:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.327+0000 7f339f59e700 1 -- start start 2023-03-03T06:26:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.327+0000 7f339f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33a006a780 0x7f33a00cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.327+0000 7f339f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33a00cb9a0 0x7f33a0158400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.327+0000 7f339f59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a015c230 con 0x7f33a0158940 2023-03-03T06:26:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.327+0000 7f339f59e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a00caac0 con 0x7f33a00cb9a0 2023-03-03T06:26:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.327+0000 7f339f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a00cd130 con 0x7f33a006a780 2023-03-03T06:26:05.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.327+0000 7f339e59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33a006a780 0x7f33a00cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:05.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.327+0000 7f339ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f33a0158940 0x7f33a00709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43336/0 (socket says 172.21.15.3:43336) 2023-03-03T06:26:05.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.327+0000 7f339dd9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33a00cb9a0 0x7f33a0158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:05.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.327+0000 7f339ed9d700 1 -- 172.21.15.3:0/3583808661 learned_addr learned my addr 172.21.15.3:0/3583808661 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:05.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.331+0000 7f339e59c700 1 -- 172.21.15.3:0/3583808661 >> v2:172.21.15.3:3300/0 conn(0x7f33a00cb9a0 msgr2=0x7f33a0158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:05.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.331+0000 7f339e59c700 1 --2- 172.21.15.3:0/3583808661 >> v2:172.21.15.3:3300/0 conn(0x7f33a00cb9a0 0x7f33a0158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.331+0000 7f339e59c700 1 -- 172.21.15.3:0/3583808661 >> v1:172.21.15.3:6789/0 conn(0x7f33a0158940 legacy=0x7f33a00709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:05.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.331+0000 7f339e59c700 1 -- 172.21.15.3:0/3583808661 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a00cbf60 con 0x7f33a006a780 2023-03-03T06:26:05.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.331+0000 7f339dd9b700 1 --2- 172.21.15.3:0/3583808661 >> v2:172.21.15.3:3300/0 conn(0x7f33a00cb9a0 0x7f33a0158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:05.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.331+0000 7f339e59c700 1 --2- 172.21.15.3:0/3583808661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a006a780 0x7f33a00cace0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f33a006f660 tx=0x7f339000bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb8b33f0f1d5efd server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:05.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.331+0000 7f339d59a700 1 -- 172.21.15.3:0/3583808661 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339000dc70 con 0x7f33a006a780 2023-03-03T06:26:05.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.331+0000 7f339d59a700 1 -- 172.21.15.3:0/3583808661 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339000ddd0 con 0x7f33a006a780 2023-03-03T06:26:05.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.331+0000 7f339d59a700 1 -- 172.21.15.3:0/3583808661 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390010690 con 0x7f33a006a780 2023-03-03T06:26:05.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.331+0000 7f339f59e700 1 -- 172.21.15.3:0/3583808661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a006a780 msgr2=0x7f33a00cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:05.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.331+0000 7f339f59e700 1 --2- 172.21.15.3:0/3583808661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a006a780 0x7f33a00cace0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f33a006f660 tx=0x7f339000bdd0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.335+0000 7f339f59e700 1 -- 172.21.15.3:0/3583808661 shutdown_connections 2023-03-03T06:26:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.335+0000 7f339f59e700 1 --2- 172.21.15.3:0/3583808661 >> v2:172.21.15.3:3300/0 conn(0x7f33a00cb9a0 0x7f33a0158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.335+0000 7f339f59e700 1 --2- 172.21.15.3:0/3583808661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a006a780 0x7f33a00cace0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.335+0000 7f339f59e700 1 -- 172.21.15.3:0/3583808661 >> 172.21.15.3:0/3583808661 conn(0x7f33a00607c0 msgr2=0x7f33a0060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.335+0000 7f339f59e700 1 -- 172.21.15.3:0/3583808661 shutdown_connections 2023-03-03T06:26:05.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.335+0000 7f339f59e700 1 -- 172.21.15.3:0/3583808661 wait complete. 2023-03-03T06:26:05.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339f59e700 1 Processor -- start 2023-03-03T06:26:05.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339f59e700 1 -- start start 2023-03-03T06:26:05.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f33a006a780 0x7f33a014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:05.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a00cb9a0 0x7f33a00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:05.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0158940 0x7f33a00aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:05.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a01eeba0 con 0x7f33a0158940 2023-03-03T06:26:05.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0150c90 con 0x7f33a00cb9a0 2023-03-03T06:26:05.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339f59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0150f70 con 0x7f33a006a780 2023-03-03T06:26:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0158940 0x7f33a00aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:05.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0158940 0x7f33a00aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59082/0 (socket says 172.21.15.3:59082) 2023-03-03T06:26:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339ed9d700 1 -- 172.21.15.3:0/1435801281 learned_addr learned my addr 172.21.15.3:0/1435801281 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339e59c700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f33a006a780 0x7f33a014cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339dd9b700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a00cb9a0 0x7f33a00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339ed9d700 1 -- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f33a006a780 msgr2=0x7f33a014cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339ed9d700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f33a006a780 0x7f33a014cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339ed9d700 1 -- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a00cb9a0 msgr2=0x7f33a00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339ed9d700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a00cb9a0 0x7f33a00ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339ed9d700 1 -- 172.21.15.3:0/1435801281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33940094b0 con 0x7f33a0158940 2023-03-03T06:26:05.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.339+0000 7f339ed9d700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0158940 0x7f33a00aebd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f339800ca80 tx=0x7f339800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:05.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.343+0000 7f338effd700 1 -- 172.21.15.3:0/1435801281 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398007480 con 0x7f33a0158940 2023-03-03T06:26:05.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.343+0000 7f338effd700 1 -- 172.21.15.3:0/1435801281 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33980075e0 con 0x7f33a0158940 2023-03-03T06:26:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.343+0000 7f339f59e700 1 -- 172.21.15.3:0/1435801281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339000b750 con 0x7f33a0158940 2023-03-03T06:26:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.343+0000 7f338effd700 1 -- 172.21.15.3:0/1435801281 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398007950 con 0x7f33a0158940 2023-03-03T06:26:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.343+0000 7f339f59e700 1 -- 172.21.15.3:0/1435801281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a015c7d0 con 0x7f33a0158940 2023-03-03T06:26:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.343+0000 7f339f59e700 1 -- 172.21.15.3:0/1435801281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a015c4f0 con 0x7f33a0158940 2023-03-03T06:26:05.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.347+0000 7f338effd700 1 -- 172.21.15.3:0/1435801281 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3398019030 con 0x7f33a0158940 2023-03-03T06:26:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.347+0000 7f338effd700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3380064ee0 0x7f3380067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.347+0000 7f338effd700 1 -- 172.21.15.3:0/1435801281 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3398081980 con 0x7f33a0158940 2023-03-03T06:26:05.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.347+0000 7f339e59c700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3380064ee0 0x7f3380067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:05.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.351+0000 7f339e59c700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3380064ee0 0x7f3380067390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3390000c00 tx=0x7f3390009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.355+0000 7f338effd700 1 -- 172.21.15.3:0/1435801281 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f339804a400 con 0x7f33a0158940 2023-03-03T06:26:05.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.531+0000 7f339f59e700 1 -- 172.21.15.3:0/1435801281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f33a015cab0 con 0x7f33a0158940 2023-03-03T06:26:05.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.531+0000 7f338effd700 1 -- 172.21.15.3:0/1435801281 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f339804f750 con 0x7f33a0158940 2023-03-03T06:26:05.537 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:26:05.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.535+0000 7f339f59e700 1 -- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3380064ee0 msgr2=0x7f3380067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:05.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.535+0000 7f339f59e700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3380064ee0 0x7f3380067390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3390000c00 tx=0x7f3390009040 comp rx=0 tx=0).stop 2023-03-03T06:26:05.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.535+0000 7f339f59e700 1 -- 172.21.15.3:0/1435801281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0158940 msgr2=0x7f33a00aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.535+0000 7f339f59e700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0158940 0x7f33a00aebd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f339800ca80 tx=0x7f339800cab0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.535+0000 7f339f59e700 1 -- 172.21.15.3:0/1435801281 shutdown_connections 2023-03-03T06:26:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.535+0000 7f339f59e700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3380064ee0 0x7f3380067390 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.535+0000 7f339f59e700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0158940 0x7f33a00aebd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.535+0000 7f339f59e700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33a00cb9a0 0x7f33a00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.535+0000 7f339f59e700 1 --2- 172.21.15.3:0/1435801281 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f33a006a780 0x7f33a014cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.535+0000 7f339f59e700 1 -- 172.21.15.3:0/1435801281 >> 172.21.15.3:0/1435801281 conn(0x7f33a00607c0 msgr2=0x7f33a0156770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:05.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.535+0000 7f339f59e700 1 -- 172.21.15.3:0/1435801281 shutdown_connections 2023-03-03T06:26:05.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.539+0000 7f339f59e700 1 -- 172.21.15.3:0/1435801281 wait complete. 2023-03-03T06:26:05.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-03T06:26:05.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.787+0000 7f914359e700 1 Processor -- start 2023-03-03T06:26:05.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.787+0000 7f914359e700 1 -- start start 2023-03-03T06:26:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f914359e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f914415d6e0 0x7f91440cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f914359e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91440cfd80 0x7f91440d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f914359e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91440cf3b0 con 0x7f9144156ba0 2023-03-03T06:26:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f914359e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9144156f70 con 0x7f91440cfd80 2023-03-03T06:26:05.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f914359e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91440cd320 con 0x7f914415d6e0 2023-03-03T06:26:05.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f9142d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91440cfd80 0x7f91440d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:05.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f9142d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91440cfd80 0x7f91440d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59084/0 (socket says 172.21.15.3:59084) 2023-03-03T06:26:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f9142d9d700 1 -- 172.21.15.3:0/758070537 learned_addr learned my addr 172.21.15.3:0/758070537 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f913bfff700 1 --2- 172.21.15.3:0/758070537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f914415d6e0 0x7f91440cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f9142d9d700 1 -- 172.21.15.3:0/758070537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f914415d6e0 msgr2=0x7f91440cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f9142d9d700 1 --2- 172.21.15.3:0/758070537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f914415d6e0 0x7f91440cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f9142d9d700 1 -- 172.21.15.3:0/758070537 >> v1:172.21.15.3:6789/0 conn(0x7f9144156ba0 legacy=0x7f9144071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:05.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f9142d9d700 1 -- 172.21.15.3:0/758070537 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91440cd600 con 0x7f91440cfd80 2023-03-03T06:26:05.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f9142d9d700 1 --2- 172.21.15.3:0/758070537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91440cfd80 0x7f91440d0b30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f913c009e50 tx=0x7f913c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c10578259a5067 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:05.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f9141d9b700 1 -- 172.21.15.3:0/758070537 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c00e070 con 0x7f91440cfd80 2023-03-03T06:26:05.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f9141d9b700 1 -- 172.21.15.3:0/758070537 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913c003a40 con 0x7f91440cfd80 2023-03-03T06:26:05.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.791+0000 7f9141d9b700 1 -- 172.21.15.3:0/758070537 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c003df0 con 0x7f91440cfd80 2023-03-03T06:26:05.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.795+0000 7f914359e700 1 -- 172.21.15.3:0/758070537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91440cfd80 msgr2=0x7f91440d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:05.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.795+0000 7f914359e700 1 --2- 172.21.15.3:0/758070537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91440cfd80 0x7f91440d0b30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f913c009e50 tx=0x7f913c00b330 comp rx=0 tx=0).stop 2023-03-03T06:26:05.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.795+0000 7f914359e700 1 -- 172.21.15.3:0/758070537 shutdown_connections 2023-03-03T06:26:05.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.795+0000 7f914359e700 1 --2- 172.21.15.3:0/758070537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91440cfd80 0x7f91440d0b30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.795+0000 7f914359e700 1 --2- 172.21.15.3:0/758070537 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f914415d6e0 0x7f91440cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.795+0000 7f914359e700 1 -- 172.21.15.3:0/758070537 >> 172.21.15.3:0/758070537 conn(0x7f914406e5c0 msgr2=0x7f914406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:05.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.795+0000 7f914359e700 1 -- 172.21.15.3:0/758070537 shutdown_connections 2023-03-03T06:26:05.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.795+0000 7f914359e700 1 -- 172.21.15.3:0/758070537 wait complete. 2023-03-03T06:26:05.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f914359e700 1 Processor -- start 2023-03-03T06:26:05.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f914359e700 1 -- start start 2023-03-03T06:26:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f914359e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91440cfd80 0x7f91440c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f914359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9144156ba0 0x7f9144155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f914359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f914415d6e0 0x7f91440b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f914359e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91441556a0 con 0x7f9144156ba0 2023-03-03T06:26:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f914359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440c7750 con 0x7f914415d6e0 2023-03-03T06:26:05.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f914359e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91440c7a30 con 0x7f91440cfd80 2023-03-03T06:26:05.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f913bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9144156ba0 0x7f9144155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:05.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f913bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9144156ba0 0x7f9144155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59100/0 (socket says 172.21.15.3:59100) 2023-03-03T06:26:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f913bfff700 1 -- 172.21.15.3:0/3437781034 learned_addr learned my addr 172.21.15.3:0/3437781034 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f914259c700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91440cfd80 0x7f91440c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f9142d9d700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f914415d6e0 0x7f91440b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f913bfff700 1 -- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91440cfd80 msgr2=0x7f91440c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f913bfff700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91440cfd80 0x7f91440c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f913bfff700 1 -- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f914415d6e0 msgr2=0x7f91440b78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:05.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f913bfff700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f914415d6e0 0x7f91440b78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.799+0000 7f913bfff700 1 -- 172.21.15.3:0/3437781034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c009b30 con 0x7f9144156ba0 2023-03-03T06:26:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.803+0000 7f913bfff700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9144156ba0 0x7f9144155160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f912c00d800 tx=0x7f912c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:05.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.803+0000 7f913affd700 1 -- 172.21.15.3:0/3437781034 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c0125f0 con 0x7f9144156ba0 2023-03-03T06:26:05.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.803+0000 7f913affd700 1 -- 172.21.15.3:0/3437781034 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c012750 con 0x7f9144156ba0 2023-03-03T06:26:05.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.803+0000 7f913affd700 1 -- 172.21.15.3:0/3437781034 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c012ac0 con 0x7f9144156ba0 2023-03-03T06:26:05.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.803+0000 7f914359e700 1 -- 172.21.15.3:0/3437781034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91440cef40 con 0x7f9144156ba0 2023-03-03T06:26:05.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.803+0000 7f914359e700 1 -- 172.21.15.3:0/3437781034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91440cd8e0 con 0x7f9144156ba0 2023-03-03T06:26:05.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.803+0000 7f914359e700 1 -- 172.21.15.3:0/3437781034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91440cf0d0 con 0x7f9144156ba0 2023-03-03T06:26:05.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.807+0000 7f913affd700 1 -- 172.21.15.3:0/3437781034 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f912c012c20 con 0x7f9144156ba0 2023-03-03T06:26:05.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.807+0000 7f913affd700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9124064ee0 0x7f9124067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:05.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.807+0000 7f913affd700 1 -- 172.21.15.3:0/3437781034 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f912c0824e0 con 0x7f9144156ba0 2023-03-03T06:26:05.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.807+0000 7f914259c700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9124064ee0 0x7f9124067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:05.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.811+0000 7f914259c700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9124064ee0 0x7f9124067390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9134007fd0 tx=0x7f9134009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:05.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.815+0000 7f913affd700 1 -- 172.21.15.3:0/3437781034 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f912c050260 con 0x7f9144156ba0 2023-03-03T06:26:05.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.987+0000 7f914359e700 1 -- 172.21.15.3:0/3437781034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9144073e90 con 0x7f9144156ba0 2023-03-03T06:26:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.987+0000 7f913affd700 1 -- 172.21.15.3:0/3437781034 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f912c053880 con 0x7f9144156ba0 2023-03-03T06:26:05.993 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:05.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.991+0000 7f914359e700 1 -- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9124064ee0 msgr2=0x7f9124067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:05.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.991+0000 7f914359e700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9124064ee0 0x7f9124067390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9134007fd0 tx=0x7f9134009040 comp rx=0 tx=0).stop 2023-03-03T06:26:05.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.991+0000 7f914359e700 1 -- 172.21.15.3:0/3437781034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9144156ba0 msgr2=0x7f9144155160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:05.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.991+0000 7f914359e700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9144156ba0 0x7f9144155160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f912c00d800 tx=0x7f912c00db10 comp rx=0 tx=0).stop 2023-03-03T06:26:05.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.991+0000 7f914359e700 1 -- 172.21.15.3:0/3437781034 shutdown_connections 2023-03-03T06:26:05.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.991+0000 7f914359e700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9124064ee0 0x7f9124067390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.991+0000 7f914359e700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f914415d6e0 0x7f91440b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.991+0000 7f914359e700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9144156ba0 0x7f9144155160 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.991+0000 7f914359e700 1 --2- 172.21.15.3:0/3437781034 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91440cfd80 0x7f91440c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:05.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.991+0000 7f914359e700 1 -- 172.21.15.3:0/3437781034 >> 172.21.15.3:0/3437781034 conn(0x7f914406e5c0 msgr2=0x7f91440b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:05.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.991+0000 7f914359e700 1 -- 172.21.15.3:0/3437781034 shutdown_connections 2023-03-03T06:26:05.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:05.991+0000 7f914359e700 1 -- 172.21.15.3:0/3437781034 wait complete. 2023-03-03T06:26:06.031 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-03T06:26:06.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.239+0000 7f941d572700 1 Processor -- start 2023-03-03T06:26:06.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.239+0000 7f941d572700 1 -- start start 2023-03-03T06:26:06.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.239+0000 7f941d572700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94180b2530 0x7f94180b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.239+0000 7f941d572700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94180cb470 con 0x7f9418158b80 2023-03-03T06:26:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.239+0000 7f941d572700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94180cb750 con 0x7f94180b4d20 2023-03-03T06:26:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.239+0000 7f941d572700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94180b2ef0 con 0x7f94180b2530 2023-03-03T06:26:06.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.239+0000 7f9416ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9418158b80 0x7f94180d0030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43350/0 (socket says 172.21.15.3:43350) 2023-03-03T06:26:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.239+0000 7f9416ffd700 1 -- 172.21.15.3:0/969024709 learned_addr learned my addr 172.21.15.3:0/969024709 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.243+0000 7f94167fc700 1 --2- 172.21.15.3:0/969024709 >> v2:172.21.15.32:3300/0 conn(0x7f94180b2530 0x7f94180b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.243+0000 7f9415ffb700 1 -- 172.21.15.3:0/969024709 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864594537 0 0) 0x7f94180cb470 con 0x7f9418158b80 2023-03-03T06:26:06.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.243+0000 7f9415ffb700 1 -- 172.21.15.3:0/969024709 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94000057e0 con 0x7f9418158b80 2023-03-03T06:26:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.243+0000 7f94167fc700 1 -- 172.21.15.3:0/969024709 >> v1:172.21.15.32:6789/0 conn(0x7f94180b4d20 legacy=0x7f94180b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.243+0000 7f94167fc700 1 -- 172.21.15.3:0/969024709 >> v1:172.21.15.3:6789/0 conn(0x7f9418158b80 legacy=0x7f94180d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:06.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.243+0000 7f94167fc700 1 -- 172.21.15.3:0/969024709 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180cba30 con 0x7f94180b2530 2023-03-03T06:26:06.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.243+0000 7f94167fc700 1 --2- 172.21.15.3:0/969024709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94180b2530 0x7f94180b29b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f940c009850 tx=0x7f940c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a2f1cdfed65b7bb server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:06.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.243+0000 7f9415ffb700 1 -- 172.21.15.3:0/969024709 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c011830 con 0x7f94180b2530 2023-03-03T06:26:06.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.243+0000 7f9415ffb700 1 -- 172.21.15.3:0/969024709 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c011990 con 0x7f94180b2530 2023-03-03T06:26:06.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.243+0000 7f9415ffb700 1 -- 172.21.15.3:0/969024709 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c011d40 con 0x7f94180b2530 2023-03-03T06:26:06.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.243+0000 7f941d572700 1 -- 172.21.15.3:0/969024709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94180b2530 msgr2=0x7f94180b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:06.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.243+0000 7f941d572700 1 --2- 172.21.15.3:0/969024709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94180b2530 0x7f94180b29b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f940c009850 tx=0x7f940c009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.247+0000 7f941d572700 1 -- 172.21.15.3:0/969024709 shutdown_connections 2023-03-03T06:26:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.247+0000 7f941d572700 1 --2- 172.21.15.3:0/969024709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94180b2530 0x7f94180b29b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.247+0000 7f941d572700 1 -- 172.21.15.3:0/969024709 >> 172.21.15.3:0/969024709 conn(0x7f941806b0d0 msgr2=0x7f94180cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.247+0000 7f941d572700 1 -- 172.21.15.3:0/969024709 shutdown_connections 2023-03-03T06:26:06.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.247+0000 7f941d572700 1 -- 172.21.15.3:0/969024709 wait complete. 2023-03-03T06:26:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.247+0000 7f941d572700 1 Processor -- start 2023-03-03T06:26:06.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.247+0000 7f941d572700 1 -- start start 2023-03-03T06:26:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f941d572700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94180b2530 0x7f94180c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:06.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f941d572700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94180b4d20 0x7f9418147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f941d572700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9418158b80 0x7f94180aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f941d572700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9418147b40 con 0x7f94180b2530 2023-03-03T06:26:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f941d572700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180ca230 con 0x7f9418158b80 2023-03-03T06:26:06.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f941d572700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94180ca510 con 0x7f94180b4d20 2023-03-03T06:26:06.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f9416ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94180b2530 0x7f94180c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:06.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f9416ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94180b2530 0x7f94180c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59114/0 (socket says 172.21.15.3:59114) 2023-03-03T06:26:06.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f9416ffd700 1 -- 172.21.15.3:0/2035133039 learned_addr learned my addr 172.21.15.3:0/2035133039 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:06.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f94167fc700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94180b4d20 0x7f9418147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:06.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f9417fff700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9418158b80 0x7f94180aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:06.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f9416ffd700 1 -- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94180b4d20 msgr2=0x7f9418147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:06.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f9416ffd700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94180b4d20 0x7f9418147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f9416ffd700 1 -- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9418158b80 msgr2=0x7f94180aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:06.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f9416ffd700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9418158b80 0x7f94180aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f9416ffd700 1 -- 172.21.15.3:0/2035133039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c0094b0 con 0x7f94180b2530 2023-03-03T06:26:06.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f9416ffd700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94180b2530 0x7f94180c9bb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f940800c4d0 tx=0x7f940800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:06.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f93ff7fe700 1 -- 172.21.15.3:0/2035133039 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408005d90 con 0x7f94180b2530 2023-03-03T06:26:06.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f93ff7fe700 1 -- 172.21.15.3:0/2035133039 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94080070d0 con 0x7f94180b2530 2023-03-03T06:26:06.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f941d572700 1 -- 172.21.15.3:0/2035133039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94180cbff0 con 0x7f94180b2530 2023-03-03T06:26:06.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f93ff7fe700 1 -- 172.21.15.3:0/2035133039 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408007480 con 0x7f94180b2530 2023-03-03T06:26:06.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.251+0000 7f941d572700 1 -- 172.21.15.3:0/2035133039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94180cbd10 con 0x7f94180b2530 2023-03-03T06:26:06.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.255+0000 7f941d572700 1 -- 172.21.15.3:0/2035133039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9418158f50 con 0x7f94180b2530 2023-03-03T06:26:06.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.255+0000 7f93ff7fe700 1 -- 172.21.15.3:0/2035133039 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9408019030 con 0x7f94180b2530 2023-03-03T06:26:06.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.259+0000 7f93ff7fe700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9404064ee0 0x7f9404067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:06.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.259+0000 7f93ff7fe700 1 -- 172.21.15.3:0/2035133039 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9408081630 con 0x7f94180b2530 2023-03-03T06:26:06.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.259+0000 7f94167fc700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9404064ee0 0x7f9404067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:06.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.263+0000 7f94167fc700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9404064ee0 0x7f9404067390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f940c0090b0 tx=0x7f940c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:06.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.263+0000 7f93ff7fe700 1 -- 172.21.15.3:0/2035133039 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f940804f3b0 con 0x7f94180b2530 2023-03-03T06:26:06.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.447+0000 7f941d572700 1 -- 172.21.15.3:0/2035133039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f941804e180 con 0x7f94180b2530 2023-03-03T06:26:06.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.451+0000 7f93ff7fe700 1 -- 172.21.15.3:0/2035133039 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f94080529d0 con 0x7f94180b2530 2023-03-03T06:26:06.454 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-03-03T06:26:06.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.455+0000 7f941d572700 1 -- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9404064ee0 msgr2=0x7f9404067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:06.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.455+0000 7f941d572700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9404064ee0 0x7f9404067390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f940c0090b0 tx=0x7f940c008030 comp rx=0 tx=0).stop 2023-03-03T06:26:06.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.455+0000 7f941d572700 1 -- 172.21.15.3:0/2035133039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94180b2530 msgr2=0x7f94180c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:06.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.455+0000 7f941d572700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94180b2530 0x7f94180c9bb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f940800c4d0 tx=0x7f940800c7e0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.455+0000 7f941d572700 1 -- 172.21.15.3:0/2035133039 shutdown_connections 2023-03-03T06:26:06.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.455+0000 7f941d572700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9404064ee0 0x7f9404067390 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.455+0000 7f941d572700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9418158b80 0x7f94180aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.455+0000 7f941d572700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94180b4d20 0x7f9418147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.455+0000 7f941d572700 1 --2- 172.21.15.3:0/2035133039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94180b2530 0x7f94180c9bb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.455+0000 7f941d572700 1 -- 172.21.15.3:0/2035133039 >> 172.21.15.3:0/2035133039 conn(0x7f941806b0d0 msgr2=0x7f941815b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:06.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.455+0000 7f941d572700 1 -- 172.21.15.3:0/2035133039 shutdown_connections 2023-03-03T06:26:06.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.455+0000 7f941d572700 1 -- 172.21.15.3:0/2035133039 wait complete. 2023-03-03T06:26:06.492 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-03T06:26:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.699+0000 7f7766c84700 1 Processor -- start 2023-03-03T06:26:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.699+0000 7f7766c84700 1 -- start start 2023-03-03T06:26:06.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.699+0000 7f7766c84700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7760159240 0x7f77600cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:06.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.699+0000 7f7766c84700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77600cdc50 0x7f776015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:06.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.699+0000 7f7766c84700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7760070020 con 0x7f77600ce150 2023-03-03T06:26:06.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.703+0000 7f7766c84700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7760070300 con 0x7f77600cdc50 2023-03-03T06:26:06.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.703+0000 7f7766c84700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776015ce30 con 0x7f7760159240 2023-03-03T06:26:06.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.703+0000 7f7765481700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f77600cdc50 0x7f776015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:06.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.703+0000 7f7766483700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f77600ce150 0x7f77600ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43364/0 (socket says 172.21.15.3:43364) 2023-03-03T06:26:06.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.703+0000 7f7766483700 1 -- 172.21.15.3:0/3599784236 learned_addr learned my addr 172.21.15.3:0/3599784236 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:06.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.703+0000 7f7765c82700 1 --2- 172.21.15.3:0/3599784236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7760159240 0x7f77600cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.703+0000 7f7765481700 1 -- 172.21.15.3:0/3599784236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7760159240 msgr2=0x7f77600cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.703+0000 7f7765481700 1 --2- 172.21.15.3:0/3599784236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7760159240 0x7f77600cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.703+0000 7f7765481700 1 -- 172.21.15.3:0/3599784236 >> v1:172.21.15.3:6789/0 conn(0x7f77600ce150 legacy=0x7f77600ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.703+0000 7f7764c80700 1 -- 172.21.15.3:0/3599784236 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383695801 0 0) 0x7f7760070020 con 0x7f77600ce150 2023-03-03T06:26:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.703+0000 7f7765481700 1 -- 172.21.15.3:0/3599784236 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776015d110 con 0x7f77600cdc50 2023-03-03T06:26:06.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.703+0000 7f7765481700 1 --2- 172.21.15.3:0/3599784236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77600cdc50 0x7f776015d630 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7750009850 tx=0x7f7750009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d760e850d8ef360 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:06.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.707+0000 7f7764c80700 1 -- 172.21.15.3:0/3599784236 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750011830 con 0x7f77600cdc50 2023-03-03T06:26:06.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.707+0000 7f7764c80700 1 -- 172.21.15.3:0/3599784236 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7750011990 con 0x7f77600cdc50 2023-03-03T06:26:06.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.707+0000 7f7764c80700 1 -- 172.21.15.3:0/3599784236 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750011d40 con 0x7f77600cdc50 2023-03-03T06:26:06.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.707+0000 7f7766c84700 1 -- 172.21.15.3:0/3599784236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77600cdc50 msgr2=0x7f776015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:06.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.707+0000 7f7766c84700 1 --2- 172.21.15.3:0/3599784236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77600cdc50 0x7f776015d630 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7750009850 tx=0x7f7750009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:06.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.707+0000 7f7766c84700 1 -- 172.21.15.3:0/3599784236 shutdown_connections 2023-03-03T06:26:06.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.707+0000 7f7766c84700 1 --2- 172.21.15.3:0/3599784236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77600cdc50 0x7f776015d630 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.707+0000 7f7766c84700 1 --2- 172.21.15.3:0/3599784236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7760159240 0x7f77600cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.707+0000 7f7766c84700 1 -- 172.21.15.3:0/3599784236 >> 172.21.15.3:0/3599784236 conn(0x7f7760069d70 msgr2=0x7f776006a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.707+0000 7f7766c84700 1 -- 172.21.15.3:0/3599784236 shutdown_connections 2023-03-03T06:26:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.707+0000 7f7766c84700 1 -- 172.21.15.3:0/3599784236 wait complete. 2023-03-03T06:26:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.711+0000 7f7766c84700 1 Processor -- start 2023-03-03T06:26:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.711+0000 7f7766c84700 1 -- start start 2023-03-03T06:26:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.711+0000 7f7766c84700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77600cdc50 0x7f7760148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.711+0000 7f7766c84700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77600ce150 0x7f77600c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.711+0000 7f7766c84700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7760159240 0x7f77600c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.711+0000 7f7766c84700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77600c7f80 con 0x7f77600ce150 2023-03-03T06:26:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.711+0000 7f7766c84700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7760149090 con 0x7f77600cdc50 2023-03-03T06:26:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.711+0000 7f7766c84700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77600c8190 con 0x7f7760159240 2023-03-03T06:26:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.711+0000 7f7765481700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77600ce150 0x7f77600c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.711+0000 7f7765481700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77600ce150 0x7f77600c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59144/0 (socket says 172.21.15.3:59144) 2023-03-03T06:26:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.711+0000 7f7765481700 1 -- 172.21.15.3:0/1926093169 learned_addr learned my addr 172.21.15.3:0/1926093169 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.711+0000 7f7765c82700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77600cdc50 0x7f7760148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.715+0000 7f7766483700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7760159240 0x7f77600c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.715+0000 7f7765481700 1 -- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7760159240 msgr2=0x7f77600c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.715+0000 7f7765481700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7760159240 0x7f77600c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.715+0000 7f7765481700 1 -- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77600cdc50 msgr2=0x7f7760148b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.715+0000 7f7765481700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77600cdc50 0x7f7760148b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.715+0000 7f7765481700 1 -- 172.21.15.3:0/1926093169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77500094b0 con 0x7f77600ce150 2023-03-03T06:26:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.715+0000 7f7765481700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77600ce150 0x7f77600c5810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7750014040 tx=0x7f7750004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.715+0000 7f77567fc700 1 -- 172.21.15.3:0/1926093169 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750019a00 con 0x7f77600ce150 2023-03-03T06:26:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.715+0000 7f77567fc700 1 -- 172.21.15.3:0/1926093169 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775001e070 con 0x7f77600ce150 2023-03-03T06:26:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.715+0000 7f77567fc700 1 -- 172.21.15.3:0/1926093169 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750019da0 con 0x7f77600ce150 2023-03-03T06:26:06.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.715+0000 7f7766c84700 1 -- 172.21.15.3:0/1926093169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77600705e0 con 0x7f77600ce150 2023-03-03T06:26:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.715+0000 7f7766c84700 1 -- 172.21.15.3:0/1926093169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7760159610 con 0x7f77600ce150 2023-03-03T06:26:06.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.715+0000 7f7766c84700 1 -- 172.21.15.3:0/1926093169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77600cd8d0 con 0x7f77600ce150 2023-03-03T06:26:06.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.719+0000 7f77567fc700 1 -- 172.21.15.3:0/1926093169 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f775000f050 con 0x7f77600ce150 2023-03-03T06:26:06.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.719+0000 7f77567fc700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7744064ee0 0x7f7744067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.723+0000 7f77567fc700 1 -- 172.21.15.3:0/1926093169 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7750083280 con 0x7f77600ce150 2023-03-03T06:26:06.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.723+0000 7f7765c82700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7744064ee0 0x7f7744067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:06.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.723+0000 7f7765c82700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7744064ee0 0x7f7744067390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f775c0113a0 tx=0x7f775c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:06.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.727+0000 7f77567fc700 1 -- 172.21.15.3:0/1926093169 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7750051000 con 0x7f77600ce150 2023-03-03T06:26:06.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.899+0000 7f7766c84700 1 -- 172.21.15.3:0/1926093169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7760073e90 con 0x7f77600ce150 2023-03-03T06:26:06.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.903+0000 7f77567fc700 1 -- 172.21.15.3:0/1926093169 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f77500200a0 con 0x7f77600ce150 2023-03-03T06:26:06.906 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.903+0000 7f7766c84700 1 -- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7744064ee0 msgr2=0x7f7744067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.903+0000 7f7766c84700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7744064ee0 0x7f7744067390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f775c0113a0 tx=0x7f775c00b040 comp rx=0 tx=0).stop 2023-03-03T06:26:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.903+0000 7f7766c84700 1 -- 172.21.15.3:0/1926093169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77600ce150 msgr2=0x7f77600c5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:06.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.903+0000 7f7766c84700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77600ce150 0x7f77600c5810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7750014040 tx=0x7f7750004770 comp rx=0 tx=0).stop 2023-03-03T06:26:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.903+0000 7f7766c84700 1 -- 172.21.15.3:0/1926093169 shutdown_connections 2023-03-03T06:26:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.903+0000 7f7766c84700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7744064ee0 0x7f7744067390 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.903+0000 7f7766c84700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7760159240 0x7f77600c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.903+0000 7f7766c84700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f77600ce150 0x7f77600c5810 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.903+0000 7f7766c84700 1 --2- 172.21.15.3:0/1926093169 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77600cdc50 0x7f7760148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.903+0000 7f7766c84700 1 -- 172.21.15.3:0/1926093169 >> 172.21.15.3:0/1926093169 conn(0x7f7760069d70 msgr2=0x7f7760158190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.907+0000 7f7766c84700 1 -- 172.21.15.3:0/1926093169 shutdown_connections 2023-03-03T06:26:06.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:06.907+0000 7f7766c84700 1 -- 172.21.15.3:0/1926093169 wait complete. 2023-03-03T06:26:06.943 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:07.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.147+0000 7f7dc1751700 1 Processor -- start 2023-03-03T06:26:07.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.147+0000 7f7dc1751700 1 -- start start 2023-03-03T06:26:07.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dc1751700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dbc1586c0 0x7f7dbc15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:07.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dc1751700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7dbc159200 0x7f7dbc159680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:07.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dc1751700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dbc159bc0 con 0x7f7dbc06ee10 2023-03-03T06:26:07.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dc1751700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dbc158a90 con 0x7f7dbc159200 2023-03-03T06:26:07.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dc1751700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc15d0f0 con 0x7f7dbc1586c0 2023-03-03T06:26:07.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dc0f50700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7dbc159200 0x7f7dbc159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:07.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dc0f50700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7dbc159200 0x7f7dbc159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59152/0 (socket says 172.21.15.3:59152) 2023-03-03T06:26:07.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dc0f50700 1 -- 172.21.15.3:0/1636279961 learned_addr learned my addr 172.21.15.3:0/1636279961 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:07.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dbbfff700 1 --2- 172.21.15.3:0/1636279961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dbc1586c0 0x7f7dbc15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:07.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dc0f50700 1 -- 172.21.15.3:0/1636279961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dbc1586c0 msgr2=0x7f7dbc15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:07.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dc0f50700 1 --2- 172.21.15.3:0/1636279961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dbc1586c0 0x7f7dbc15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dc0f50700 1 -- 172.21.15.3:0/1636279961 >> v1:172.21.15.32:6789/0 conn(0x7f7dbc06ee10 legacy=0x7f7dbc159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:07.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dc0f50700 1 -- 172.21.15.3:0/1636279961 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0c5a20 con 0x7f7dbc159200 2023-03-03T06:26:07.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dc0f50700 1 --2- 172.21.15.3:0/1636279961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc159200 0x7f7dbc159680 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7db400af60 tx=0x7f7db400af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff762a1d4fcd1dd server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:07.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dbb7fe700 1 -- 172.21.15.3:0/1636279961 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db400f070 con 0x7f7dbc159200 2023-03-03T06:26:07.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dbb7fe700 1 -- 172.21.15.3:0/1636279961 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db400a2e0 con 0x7f7dbc159200 2023-03-03T06:26:07.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.151+0000 7f7dbb7fe700 1 -- 172.21.15.3:0/1636279961 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4003c80 con 0x7f7dbc159200 2023-03-03T06:26:07.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.155+0000 7f7dc1751700 1 -- 172.21.15.3:0/1636279961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc159200 msgr2=0x7f7dbc159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:07.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.155+0000 7f7dc1751700 1 --2- 172.21.15.3:0/1636279961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc159200 0x7f7dbc159680 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7db400af60 tx=0x7f7db400af90 comp rx=0 tx=0).stop 2023-03-03T06:26:07.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.155+0000 7f7dc1751700 1 -- 172.21.15.3:0/1636279961 shutdown_connections 2023-03-03T06:26:07.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.155+0000 7f7dc1751700 1 --2- 172.21.15.3:0/1636279961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc159200 0x7f7dbc159680 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.155+0000 7f7dc1751700 1 --2- 172.21.15.3:0/1636279961 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dbc1586c0 0x7f7dbc15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.155+0000 7f7dc1751700 1 -- 172.21.15.3:0/1636279961 >> 172.21.15.3:0/1636279961 conn(0x7f7dbc071b30 msgr2=0x7f7dbc0b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:07.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.155+0000 7f7dc1751700 1 -- 172.21.15.3:0/1636279961 shutdown_connections 2023-03-03T06:26:07.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.155+0000 7f7dc1751700 1 -- 172.21.15.3:0/1636279961 wait complete. 2023-03-03T06:26:07.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.155+0000 7f7dc1751700 1 Processor -- start 2023-03-03T06:26:07.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7dc1751700 1 -- start start 2023-03-03T06:26:07.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7dc1751700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc06ee10 0x7f7dbc152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:07.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7dc1751700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc1586c0 0x7f7dbc15b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:07.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7dc1751700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dbc159200 0x7f7dbc15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:07.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7dc1751700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc0b2960 con 0x7f7dbc1586c0 2023-03-03T06:26:07.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7dc1751700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc0b1ba0 con 0x7f7dbc06ee10 2023-03-03T06:26:07.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7dc1751700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc0b1e80 con 0x7f7dbc159200 2023-03-03T06:26:07.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7db37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc1586c0 0x7f7dbc15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:07.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7db37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc1586c0 0x7f7dbc15b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59166/0 (socket says 172.21.15.3:59166) 2023-03-03T06:26:07.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7db37fe700 1 -- 172.21.15.3:0/1238299601 learned_addr learned my addr 172.21.15.3:0/1238299601 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:07.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7dc0f50700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dbc159200 0x7f7dbc15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:07.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7dbbfff700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc06ee10 0x7f7dbc152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:07.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7db37fe700 1 -- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dbc159200 msgr2=0x7f7dbc15b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:07.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7db37fe700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dbc159200 0x7f7dbc15b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7db37fe700 1 -- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc06ee10 msgr2=0x7f7dbc152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:07.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7db37fe700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc06ee10 0x7f7dbc152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7db37fe700 1 -- 172.21.15.3:0/1238299601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db400ac40 con 0x7f7dbc1586c0 2023-03-03T06:26:07.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.159+0000 7f7db37fe700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc1586c0 0x7f7dbc15b090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7da400b4a0 tx=0x7f7da400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:07.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.163+0000 7f7db97fa700 1 -- 172.21.15.3:0/1238299601 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4011830 con 0x7f7dbc1586c0 2023-03-03T06:26:07.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.163+0000 7f7db97fa700 1 -- 172.21.15.3:0/1238299601 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da4011990 con 0x7f7dbc1586c0 2023-03-03T06:26:07.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.163+0000 7f7dc1751700 1 -- 172.21.15.3:0/1238299601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dbc15cb30 con 0x7f7dbc1586c0 2023-03-03T06:26:07.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.163+0000 7f7db97fa700 1 -- 172.21.15.3:0/1238299601 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4011d40 con 0x7f7dbc1586c0 2023-03-03T06:26:07.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.163+0000 7f7dc1751700 1 -- 172.21.15.3:0/1238299601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dbc15c850 con 0x7f7dbc1586c0 2023-03-03T06:26:07.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.163+0000 7f7dc1751700 1 -- 172.21.15.3:0/1238299601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dbc15ce10 con 0x7f7dbc1586c0 2023-03-03T06:26:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.167+0000 7f7db97fa700 1 -- 172.21.15.3:0/1238299601 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7da4021070 con 0x7f7dbc1586c0 2023-03-03T06:26:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.167+0000 7f7db97fa700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d9c064ee0 0x7f7d9c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:07.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.171+0000 7f7db97fa700 1 -- 172.21.15.3:0/1238299601 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7da40820f0 con 0x7f7dbc1586c0 2023-03-03T06:26:07.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.171+0000 7f7dbbfff700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d9c064ee0 0x7f7d9c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:07.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.171+0000 7f7dbbfff700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d9c064ee0 0x7f7d9c067390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc153ab0 tx=0x7f7dac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:07.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.175+0000 7f7db97fa700 1 -- 172.21.15.3:0/1238299601 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7da404fe70 con 0x7f7dbc1586c0 2023-03-03T06:26:07.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.351+0000 7f7dc1751700 1 -- 172.21.15.3:0/1238299601 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dbc073e90 con 0x7f7dbc1586c0 2023-03-03T06:26:07.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.355+0000 7f7db97fa700 1 -- 172.21.15.3:0/1238299601 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f7da4053490 con 0x7f7dbc1586c0 2023-03-03T06:26:07.358 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:07.359 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:53.877945+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:07.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.355+0000 7f7dc1751700 1 -- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d9c064ee0 msgr2=0x7f7d9c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:07.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.355+0000 7f7dc1751700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d9c064ee0 0x7f7d9c067390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc153ab0 tx=0x7f7dac008040 comp rx=0 tx=0).stop 2023-03-03T06:26:07.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.355+0000 7f7dc1751700 1 -- 172.21.15.3:0/1238299601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc1586c0 msgr2=0x7f7dbc15b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:07.361 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.355+0000 7f7dc1751700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc1586c0 0x7f7dbc15b090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7da400b4a0 tx=0x7f7da400b7b0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.359+0000 7f7dc1751700 1 -- 172.21.15.3:0/1238299601 shutdown_connections 2023-03-03T06:26:07.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.359+0000 7f7dc1751700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7d9c064ee0 0x7f7d9c067390 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.359+0000 7f7dc1751700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7dbc159200 0x7f7dbc15b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.359+0000 7f7dc1751700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7dbc1586c0 0x7f7dbc15b090 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.359+0000 7f7dc1751700 1 --2- 172.21.15.3:0/1238299601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dbc06ee10 0x7f7dbc152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.359+0000 7f7dc1751700 1 -- 172.21.15.3:0/1238299601 >> 172.21.15.3:0/1238299601 conn(0x7f7dbc071b30 msgr2=0x7f7dbc156330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:07.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.359+0000 7f7dc1751700 1 -- 172.21.15.3:0/1238299601 shutdown_connections 2023-03-03T06:26:07.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.359+0000 7f7dc1751700 1 -- 172.21.15.3:0/1238299601 wait complete. 2023-03-03T06:26:07.396 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-03T06:25:11.468534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-03T06:25:12.813283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-03T06:25:16.382800+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-03T06:26:07.397 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:26:07.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.619+0000 7f627d4dc700 1 Processor -- start 2023-03-03T06:26:07.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.619+0000 7f627d4dc700 1 -- start start 2023-03-03T06:26:07.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f627d4dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f62780ce720 0x7f62780ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:07.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f627d4dc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62780cae40 0x7f62780cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:07.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f627d4dc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62780c9510 con 0x7f627806e5c0 2023-03-03T06:26:07.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f627d4dc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62780ca960 con 0x7f62780cae40 2023-03-03T06:26:07.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f627d4dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62780cf030 con 0x7f62780ce720 2023-03-03T06:26:07.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f62777fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f627806e5c0 0x7f62780c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43370/0 (socket says 172.21.15.3:43370) 2023-03-03T06:26:07.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f62777fe700 1 -- 172.21.15.3:0/1521948894 learned_addr learned my addr 172.21.15.3:0/1521948894 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:07.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f6277fff700 1 --2- 172.21.15.3:0/1521948894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f62780ce720 0x7f62780ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:07.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f627ccdb700 1 --2- 172.21.15.3:0/1521948894 >> v2:172.21.15.32:3300/0 conn(0x7f62780cae40 0x7f62780cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:07.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f6276ffd700 1 -- 172.21.15.3:0/1521948894 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146187325 0 0) 0x7f62780c9510 con 0x7f627806e5c0 2023-03-03T06:26:07.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f6276ffd700 1 -- 172.21.15.3:0/1521948894 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62600035d0 con 0x7f627806e5c0 2023-03-03T06:26:07.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f6277fff700 1 -- 172.21.15.3:0/1521948894 >> v2:172.21.15.32:3300/0 conn(0x7f62780cae40 msgr2=0x7f62780cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:07.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f6277fff700 1 --2- 172.21.15.3:0/1521948894 >> v2:172.21.15.32:3300/0 conn(0x7f62780cae40 0x7f62780cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f6277fff700 1 -- 172.21.15.3:0/1521948894 >> v1:172.21.15.3:6789/0 conn(0x7f627806e5c0 legacy=0x7f62780c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:07.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.623+0000 7f6277fff700 1 -- 172.21.15.3:0/1521948894 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627806eac0 con 0x7f62780ce720 2023-03-03T06:26:07.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.627+0000 7f6277fff700 1 --2- 172.21.15.3:0/1521948894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f62780ce720 0x7f62780ceaf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6268009850 tx=0x7f6268009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fcb15fb3cf47fe0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:07.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.627+0000 7f6276ffd700 1 -- 172.21.15.3:0/1521948894 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268011830 con 0x7f62780ce720 2023-03-03T06:26:07.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.627+0000 7f6276ffd700 1 -- 172.21.15.3:0/1521948894 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6268011990 con 0x7f62780ce720 2023-03-03T06:26:07.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.627+0000 7f6276ffd700 1 -- 172.21.15.3:0/1521948894 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268011d40 con 0x7f62780ce720 2023-03-03T06:26:07.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.627+0000 7f627d4dc700 1 -- 172.21.15.3:0/1521948894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f62780ce720 msgr2=0x7f62780ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:07.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.627+0000 7f627d4dc700 1 --2- 172.21.15.3:0/1521948894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f62780ce720 0x7f62780ceaf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6268009850 tx=0x7f6268009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:07.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.627+0000 7f627d4dc700 1 -- 172.21.15.3:0/1521948894 shutdown_connections 2023-03-03T06:26:07.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.627+0000 7f627d4dc700 1 --2- 172.21.15.3:0/1521948894 >> v2:172.21.15.32:3300/0 conn(0x7f62780cae40 0x7f62780cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.627+0000 7f627d4dc700 1 --2- 172.21.15.3:0/1521948894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f62780ce720 0x7f62780ceaf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.627+0000 7f627d4dc700 1 -- 172.21.15.3:0/1521948894 >> 172.21.15.3:0/1521948894 conn(0x7f62780607c0 msgr2=0x7f6278060b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:07.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.627+0000 7f627d4dc700 1 -- 172.21.15.3:0/1521948894 shutdown_connections 2023-03-03T06:26:07.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.627+0000 7f627d4dc700 1 -- 172.21.15.3:0/1521948894 wait complete. 2023-03-03T06:26:07.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f627d4dc700 1 Processor -- start 2023-03-03T06:26:07.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f627d4dc700 1 -- start start 2023-03-03T06:26:07.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f627d4dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627806e5c0 0x7f6278159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:07.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f627d4dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62780cae40 0x7f627815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:07.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f627d4dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f62780ce720 0x7f62780cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:07.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f627d4dc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627815bed0 con 0x7f627806e5c0 2023-03-03T06:26:07.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f627d4dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6278159680 con 0x7f62780cae40 2023-03-03T06:26:07.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f627d4dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627815b350 con 0x7f62780ce720 2023-03-03T06:26:07.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f6277fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627806e5c0 0x7f6278159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:07.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f6277fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627806e5c0 0x7f6278159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59168/0 (socket says 172.21.15.3:59168) 2023-03-03T06:26:07.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f6277fff700 1 -- 172.21.15.3:0/3564047950 learned_addr learned my addr 172.21.15.3:0/3564047950 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:07.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f62777fe700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62780cae40 0x7f627815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:07.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f627ccdb700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f62780ce720 0x7f62780cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:07.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f6277fff700 1 -- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f62780ce720 msgr2=0x7f62780cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:07.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f6277fff700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f62780ce720 0x7f62780cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f6277fff700 1 -- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62780cae40 msgr2=0x7f627815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:07.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f6277fff700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62780cae40 0x7f627815a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f6277fff700 1 -- 172.21.15.3:0/3564047950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62680094b0 con 0x7f627806e5c0 2023-03-03T06:26:07.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f627ccdb700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f62780ce720 0x7f62780cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:07.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.631+0000 7f6277fff700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627806e5c0 0x7f6278159140 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f62680090b0 tx=0x7f6268004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:07.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.635+0000 7f6274ff9700 1 -- 172.21.15.3:0/3564047950 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268004520 con 0x7f627806e5c0 2023-03-03T06:26:07.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.635+0000 7f6274ff9700 1 -- 172.21.15.3:0/3564047950 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626801f070 con 0x7f627806e5c0 2023-03-03T06:26:07.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.635+0000 7f6274ff9700 1 -- 172.21.15.3:0/3564047950 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268003900 con 0x7f627806e5c0 2023-03-03T06:26:07.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.635+0000 7f627d4dc700 1 -- 172.21.15.3:0/3564047950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627806f080 con 0x7f627806e5c0 2023-03-03T06:26:07.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.635+0000 7f627d4dc700 1 -- 172.21.15.3:0/3564047950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627806eda0 con 0x7f627806e5c0 2023-03-03T06:26:07.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.635+0000 7f627d4dc700 1 -- 172.21.15.3:0/3564047950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627806f360 con 0x7f627806e5c0 2023-03-03T06:26:07.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.639+0000 7f6274ff9700 1 -- 172.21.15.3:0/3564047950 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f626800f050 con 0x7f627806e5c0 2023-03-03T06:26:07.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.639+0000 7f6274ff9700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6264064ee0 0x7f6264067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:07.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.639+0000 7f6274ff9700 1 -- 172.21.15.3:0/3564047950 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6268021070 con 0x7f627806e5c0 2023-03-03T06:26:07.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.639+0000 7f62777fe700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6264064ee0 0x7f6264067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:07.646 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.643+0000 7f62777fe700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6264064ee0 0x7f6264067390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f62780af7b0 tx=0x7f626c007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:07.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.643+0000 7f6274ff9700 1 -- 172.21.15.3:0/3564047950 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f62680514d0 con 0x7f627806e5c0 2023-03-03T06:26:07.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.819+0000 7f627d4dc700 1 -- 172.21.15.3:0/3564047950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6278073e90 con 0x7f627806e5c0 2023-03-03T06:26:07.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.819+0000 7f6274ff9700 1 -- 172.21.15.3:0/3564047950 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f62680150a0 con 0x7f627806e5c0 2023-03-03T06:26:07.823 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:26:07.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.823+0000 7f627d4dc700 1 -- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6264064ee0 msgr2=0x7f6264067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:07.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.823+0000 7f627d4dc700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6264064ee0 0x7f6264067390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f62780af7b0 tx=0x7f626c007de0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.823+0000 7f627d4dc700 1 -- 172.21.15.3:0/3564047950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627806e5c0 msgr2=0x7f6278159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:07.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.823+0000 7f627d4dc700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627806e5c0 0x7f6278159140 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f62680090b0 tx=0x7f6268004770 comp rx=0 tx=0).stop 2023-03-03T06:26:07.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.823+0000 7f627d4dc700 1 -- 172.21.15.3:0/3564047950 shutdown_connections 2023-03-03T06:26:07.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.823+0000 7f627d4dc700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6264064ee0 0x7f6264067390 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.823+0000 7f627d4dc700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f62780ce720 0x7f62780cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.823+0000 7f627d4dc700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62780cae40 0x7f627815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.823+0000 7f627d4dc700 1 --2- 172.21.15.3:0/3564047950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f627806e5c0 0x7f6278159140 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:07.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.823+0000 7f627d4dc700 1 -- 172.21.15.3:0/3564047950 >> 172.21.15.3:0/3564047950 conn(0x7f62780607c0 msgr2=0x7f62780b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:07.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.823+0000 7f627d4dc700 1 -- 172.21.15.3:0/3564047950 shutdown_connections 2023-03-03T06:26:07.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:07.823+0000 7f627d4dc700 1 -- 172.21.15.3:0/3564047950 wait complete. 2023-03-03T06:26:07.859 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:26:08.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.067+0000 7f4815a20700 1 Processor -- start 2023-03-03T06:26:08.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.067+0000 7f4815a20700 1 -- start start 2023-03-03T06:26:08.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.071+0000 7f4815a20700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4810156cc0 0x7f48100713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f4815a20700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48100cf610 0x7f48100cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f4815a20700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48100cffd0 con 0x7f481015d720 2023-03-03T06:26:08.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f4815a20700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4810160fc0 con 0x7f48100cf610 2023-03-03T06:26:08.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f4815a20700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48101600f0 con 0x7f4810156cc0 2023-03-03T06:26:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f4814a1e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4810156cc0 0x7f48100713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f4814a1e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4810156cc0 0x7f48100713a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:60576/0 (socket says 172.21.15.3:60576) 2023-03-03T06:26:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f4814a1e700 1 -- 172.21.15.3:0/1861842874 learned_addr learned my addr 172.21.15.3:0/1861842874 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f481521f700 1 --2- 172.21.15.3:0/1861842874 >> v2:172.21.15.32:3300/0 conn(0x7f48100cf610 0x7f48100cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f4814a1e700 1 -- 172.21.15.3:0/1861842874 >> v2:172.21.15.32:3300/0 conn(0x7f48100cf610 msgr2=0x7f48100cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f4814a1e700 1 --2- 172.21.15.3:0/1861842874 >> v2:172.21.15.32:3300/0 conn(0x7f48100cf610 0x7f48100cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f4814a1e700 1 -- 172.21.15.3:0/1861842874 >> v1:172.21.15.32:6789/0 conn(0x7f481015d720 legacy=0x7f48101608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f4814a1e700 1 -- 172.21.15.3:0/1861842874 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48100718e0 con 0x7f4810156cc0 2023-03-03T06:26:08.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f481521f700 1 --2- 172.21.15.3:0/1861842874 >> v2:172.21.15.32:3300/0 conn(0x7f48100cf610 0x7f48100cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:08.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f480f7fe700 1 -- 172.21.15.3:0/1861842874 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427086449 0 0) 0x7f47fc001110 con 0x7f481015d720 2023-03-03T06:26:08.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.075+0000 7f4814a1e700 1 --2- 172.21.15.3:0/1861842874 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4810156cc0 0x7f48100713a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4804009e50 tx=0x7f480400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1eed139041b58c5 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.079+0000 7f480f7fe700 1 -- 172.21.15.3:0/1861842874 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480400e070 con 0x7f4810156cc0 2023-03-03T06:26:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.079+0000 7f480f7fe700 1 -- 172.21.15.3:0/1861842874 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480400b9c0 con 0x7f4810156cc0 2023-03-03T06:26:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.079+0000 7f480f7fe700 1 -- 172.21.15.3:0/1861842874 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480400bd70 con 0x7f4810156cc0 2023-03-03T06:26:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.079+0000 7f4815a20700 1 -- 172.21.15.3:0/1861842874 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4810156cc0 msgr2=0x7f48100713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.079+0000 7f4815a20700 1 --2- 172.21.15.3:0/1861842874 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4810156cc0 0x7f48100713a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4804009e50 tx=0x7f480400b330 comp rx=0 tx=0).stop 2023-03-03T06:26:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.079+0000 7f4815a20700 1 -- 172.21.15.3:0/1861842874 shutdown_connections 2023-03-03T06:26:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.079+0000 7f4815a20700 1 --2- 172.21.15.3:0/1861842874 >> v2:172.21.15.32:3300/0 conn(0x7f48100cf610 0x7f48100cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.079+0000 7f4815a20700 1 --2- 172.21.15.3:0/1861842874 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4810156cc0 0x7f48100713a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.079+0000 7f4815a20700 1 -- 172.21.15.3:0/1861842874 >> 172.21.15.3:0/1861842874 conn(0x7f481006e5c0 msgr2=0x7f481006e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:08.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.079+0000 7f4815a20700 1 -- 172.21.15.3:0/1861842874 shutdown_connections 2023-03-03T06:26:08.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.079+0000 7f4815a20700 1 -- 172.21.15.3:0/1861842874 wait complete. 2023-03-03T06:26:08.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.079+0000 7f4815a20700 1 Processor -- start 2023-03-03T06:26:08.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.079+0000 7f4815a20700 1 -- start start 2023-03-03T06:26:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f4815a20700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48100cf610 0x7f48100d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f4815a20700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4810156cc0 0x7f48100cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f4815a20700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f481015d720 0x7f481014b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f4815a20700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48100c9d00 con 0x7f4810156cc0 2023-03-03T06:26:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f4815a20700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48100c9f50 con 0x7f481015d720 2023-03-03T06:26:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f4815a20700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48100ca230 con 0x7f48100cf610 2023-03-03T06:26:08.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f480ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4810156cc0 0x7f48100cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f480ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4810156cc0 0x7f48100cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59170/0 (socket says 172.21.15.3:59170) 2023-03-03T06:26:08.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f480ffff700 1 -- 172.21.15.3:0/2949865384 learned_addr learned my addr 172.21.15.3:0/2949865384 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:08.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f481521f700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f481015d720 0x7f481014b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f4814a1e700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48100cf610 0x7f48100d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f481521f700 1 -- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48100cf610 msgr2=0x7f48100d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f481521f700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48100cf610 0x7f48100d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f481521f700 1 -- 172.21.15.3:0/2949865384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4810156cc0 msgr2=0x7f48100cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f481521f700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4810156cc0 0x7f48100cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f481521f700 1 -- 172.21.15.3:0/2949865384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4804009b30 con 0x7f481015d720 2023-03-03T06:26:08.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.083+0000 7f480ffff700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4810156cc0 0x7f48100cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:08.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.087+0000 7f481521f700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f481015d720 0x7f481014b360 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f480800e570 tx=0x7f480800e880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.087+0000 7f480d7fa700 1 -- 172.21.15.3:0/2949865384 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808013610 con 0x7f481015d720 2023-03-03T06:26:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.087+0000 7f480d7fa700 1 -- 172.21.15.3:0/2949865384 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4808013770 con 0x7f481015d720 2023-03-03T06:26:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.087+0000 7f480d7fa700 1 -- 172.21.15.3:0/2949865384 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808013b00 con 0x7f481015d720 2023-03-03T06:26:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.087+0000 7f4815a20700 1 -- 172.21.15.3:0/2949865384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48100cd700 con 0x7f481015d720 2023-03-03T06:26:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.087+0000 7f4815a20700 1 -- 172.21.15.3:0/2949865384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48100cd420 con 0x7f481015d720 2023-03-03T06:26:08.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.087+0000 7f4815a20700 1 -- 172.21.15.3:0/2949865384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4810160390 con 0x7f481015d720 2023-03-03T06:26:08.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.091+0000 7f480d7fa700 1 -- 172.21.15.3:0/2949865384 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4808019030 con 0x7f481015d720 2023-03-03T06:26:08.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.095+0000 7f480d7fa700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f47f4064ee0 0x7f47f4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.095+0000 7f480d7fa700 1 -- 172.21.15.3:0/2949865384 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f48080818c0 con 0x7f481015d720 2023-03-03T06:26:08.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.095+0000 7f4814a1e700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f47f4064ee0 0x7f47f4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.095+0000 7f4814a1e700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f47f4064ee0 0x7f47f4067390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f480400b370 tx=0x7f4804009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:08.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.099+0000 7f480d7fa700 1 -- 172.21.15.3:0/2949865384 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f480804f5e0 con 0x7f481015d720 2023-03-03T06:26:08.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.275+0000 7f4815a20700 1 -- 172.21.15.3:0/2949865384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4810073e90 con 0x7f481015d720 2023-03-03T06:26:08.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.279+0000 7f480d7fa700 1 -- 172.21.15.3:0/2949865384 <== mon.1 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4808052c00 con 0x7f481015d720 2023-03-03T06:26:08.282 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:26:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.283+0000 7f4815a20700 1 -- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f47f4064ee0 msgr2=0x7f47f4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.283+0000 7f4815a20700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f47f4064ee0 0x7f47f4067390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f480400b370 tx=0x7f4804009000 comp rx=0 tx=0).stop 2023-03-03T06:26:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.283+0000 7f4815a20700 1 -- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f481015d720 msgr2=0x7f481014b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.283+0000 7f4815a20700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f481015d720 0x7f481014b360 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f480800e570 tx=0x7f480800e880 comp rx=0 tx=0).stop 2023-03-03T06:26:08.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.283+0000 7f4815a20700 1 -- 172.21.15.3:0/2949865384 shutdown_connections 2023-03-03T06:26:08.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.283+0000 7f4815a20700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f47f4064ee0 0x7f47f4067390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.283+0000 7f4815a20700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f481015d720 0x7f481014b360 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.283+0000 7f4815a20700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4810156cc0 0x7f48100cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.283+0000 7f4815a20700 1 --2- 172.21.15.3:0/2949865384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48100cf610 0x7f48100d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.283+0000 7f4815a20700 1 -- 172.21.15.3:0/2949865384 >> 172.21.15.3:0/2949865384 conn(0x7f481006e5c0 msgr2=0x7f481015ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:08.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.283+0000 7f4815a20700 1 -- 172.21.15.3:0/2949865384 shutdown_connections 2023-03-03T06:26:08.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.283+0000 7f4815a20700 1 -- 172.21.15.3:0/2949865384 wait complete. 2023-03-03T06:26:08.319 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-03T06:26:08.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.527+0000 7f86e51ef700 1 Processor -- start 2023-03-03T06:26:08.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.527+0000 7f86e51ef700 1 -- start start 2023-03-03T06:26:08.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86e51ef700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86e00caab0 0x7f86e00c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86e51ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86e00b1850 0x7f86e00b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86e51ef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e006d750 con 0x7f86e00b07c0 2023-03-03T06:26:08.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86e51ef700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e015a640 con 0x7f86e00caab0 2023-03-03T06:26:08.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86e51ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e006cb40 con 0x7f86e00b1850 2023-03-03T06:26:08.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86df7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86e00caab0 0x7f86e00c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86df7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f86e00caab0 0x7f86e00c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59172/0 (socket says 172.21.15.3:59172) 2023-03-03T06:26:08.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86df7fe700 1 -- 172.21.15.3:0/227771344 learned_addr learned my addr 172.21.15.3:0/227771344 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:08.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86e49ee700 1 --2- 172.21.15.3:0/227771344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86e00b1850 0x7f86e00b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86df7fe700 1 -- 172.21.15.3:0/227771344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86e00b1850 msgr2=0x7f86e00b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:08.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86df7fe700 1 --2- 172.21.15.3:0/227771344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86e00b1850 0x7f86e00b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86df7fe700 1 -- 172.21.15.3:0/227771344 >> v1:172.21.15.32:6789/0 conn(0x7f86e00b07c0 legacy=0x7f86e00b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86df7fe700 1 -- 172.21.15.3:0/227771344 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e006ce20 con 0x7f86e00caab0 2023-03-03T06:26:08.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.531+0000 7f86df7fe700 1 --2- 172.21.15.3:0/227771344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86e00caab0 0x7f86e00c8e00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009910 tx=0x7f86d4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a86641d9015be8a server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:08.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.535+0000 7f86deffd700 1 -- 172.21.15.3:0/227771344 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4011830 con 0x7f86e00caab0 2023-03-03T06:26:08.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.535+0000 7f86deffd700 1 -- 172.21.15.3:0/227771344 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d4011990 con 0x7f86e00caab0 2023-03-03T06:26:08.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.535+0000 7f86deffd700 1 -- 172.21.15.3:0/227771344 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4011d40 con 0x7f86e00caab0 2023-03-03T06:26:08.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.535+0000 7f86e51ef700 1 -- 172.21.15.3:0/227771344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86e00caab0 msgr2=0x7f86e00c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.535+0000 7f86e51ef700 1 --2- 172.21.15.3:0/227771344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86e00caab0 0x7f86e00c8e00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009910 tx=0x7f86d4009c20 comp rx=0 tx=0).stop 2023-03-03T06:26:08.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.535+0000 7f86e51ef700 1 -- 172.21.15.3:0/227771344 shutdown_connections 2023-03-03T06:26:08.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.535+0000 7f86e51ef700 1 --2- 172.21.15.3:0/227771344 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86e00b1850 0x7f86e00b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.535+0000 7f86e51ef700 1 --2- 172.21.15.3:0/227771344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86e00caab0 0x7f86e00c8e00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.535+0000 7f86e51ef700 1 -- 172.21.15.3:0/227771344 >> 172.21.15.3:0/227771344 conn(0x7f86e00607c0 msgr2=0x7f86e0060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:08.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.535+0000 7f86e51ef700 1 -- 172.21.15.3:0/227771344 shutdown_connections 2023-03-03T06:26:08.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.535+0000 7f86e51ef700 1 -- 172.21.15.3:0/227771344 wait complete. 2023-03-03T06:26:08.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.539+0000 7f86e51ef700 1 Processor -- start 2023-03-03T06:26:08.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.539+0000 7f86e51ef700 1 -- start start 2023-03-03T06:26:08.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e51ef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86e00b07c0 0x7f86e00cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e51ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e00b1850 0x7f86e00c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e51ef700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86e00caab0 0x7f86e00aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e51ef700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e01f9dc0 con 0x7f86e00caab0 2023-03-03T06:26:08.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e51ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e01fa0a0 con 0x7f86e00b1850 2023-03-03T06:26:08.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e51ef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e01fa380 con 0x7f86e00b07c0 2023-03-03T06:26:08.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e49ee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86e00caab0 0x7f86e00aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e49ee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86e00caab0 0x7f86e00aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59174/0 (socket says 172.21.15.3:59174) 2023-03-03T06:26:08.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e49ee700 1 -- 172.21.15.3:0/2130065709 learned_addr learned my addr 172.21.15.3:0/2130065709 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:08.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86dffff700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86e00b07c0 0x7f86e00cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86df7fe700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e00b1850 0x7f86e00c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e49ee700 1 -- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86e00b07c0 msgr2=0x7f86e00cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e49ee700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86e00b07c0 0x7f86e00cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e49ee700 1 -- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e00b1850 msgr2=0x7f86e00c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e49ee700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e00b1850 0x7f86e00c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e49ee700 1 -- 172.21.15.3:0/2130065709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d4009570 con 0x7f86e00caab0 2023-03-03T06:26:08.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.543+0000 7f86e49ee700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86e00caab0 0x7f86e00aeea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f86d800efc0 tx=0x7f86d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:08.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.547+0000 7f86dcff9700 1 -- 172.21.15.3:0/2130065709 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8009ea0 con 0x7f86e00caab0 2023-03-03T06:26:08.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.547+0000 7f86dcff9700 1 -- 172.21.15.3:0/2130065709 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d800f040 con 0x7f86e00caab0 2023-03-03T06:26:08.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.547+0000 7f86e51ef700 1 -- 172.21.15.3:0/2130065709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e006d3e0 con 0x7f86e00caab0 2023-03-03T06:26:08.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.547+0000 7f86dcff9700 1 -- 172.21.15.3:0/2130065709 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d800ced0 con 0x7f86e00caab0 2023-03-03T06:26:08.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.547+0000 7f86e51ef700 1 -- 172.21.15.3:0/2130065709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e006d100 con 0x7f86e00caab0 2023-03-03T06:26:08.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.547+0000 7f86e51ef700 1 -- 172.21.15.3:0/2130065709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e00adf90 con 0x7f86e00caab0 2023-03-03T06:26:08.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.551+0000 7f86dcff9700 1 -- 172.21.15.3:0/2130065709 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f86d8014480 con 0x7f86e00caab0 2023-03-03T06:26:08.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.551+0000 7f86dcff9700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f86cc064ee0 0x7f86cc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.555+0000 7f86dcff9700 1 -- 172.21.15.3:0/2130065709 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f86d8082bf0 con 0x7f86e00caab0 2023-03-03T06:26:08.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.555+0000 7f86dffff700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f86cc064ee0 0x7f86cc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.555+0000 7f86dffff700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f86cc064ee0 0x7f86cc067390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f86d0000f60 tx=0x7f86d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:08.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.559+0000 7f86dcff9700 1 -- 172.21.15.3:0/2130065709 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f86d8050910 con 0x7f86e00caab0 2023-03-03T06:26:08.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.735+0000 7f86e51ef700 1 -- 172.21.15.3:0/2130065709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f86e0073e90 con 0x7f86e00caab0 2023-03-03T06:26:08.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.735+0000 7f86dcff9700 1 -- 172.21.15.3:0/2130065709 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f86d8053f30 con 0x7f86e00caab0 2023-03-03T06:26:08.740 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:08.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.739+0000 7f86e51ef700 1 -- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f86cc064ee0 msgr2=0x7f86cc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.739+0000 7f86e51ef700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f86cc064ee0 0x7f86cc067390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f86d0000f60 tx=0x7f86d0009040 comp rx=0 tx=0).stop 2023-03-03T06:26:08.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.739+0000 7f86e51ef700 1 -- 172.21.15.3:0/2130065709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86e00caab0 msgr2=0x7f86e00aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.739+0000 7f86e51ef700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86e00caab0 0x7f86e00aeea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f86d800efc0 tx=0x7f86d800c3a0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.739+0000 7f86e51ef700 1 -- 172.21.15.3:0/2130065709 shutdown_connections 2023-03-03T06:26:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.739+0000 7f86e51ef700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f86cc064ee0 0x7f86cc067390 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.739+0000 7f86e51ef700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86e00caab0 0x7f86e00aeea0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.739+0000 7f86e51ef700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86e00b1850 0x7f86e00c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.739+0000 7f86e51ef700 1 --2- 172.21.15.3:0/2130065709 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f86e00b07c0 0x7f86e00cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.739+0000 7f86e51ef700 1 -- 172.21.15.3:0/2130065709 >> 172.21.15.3:0/2130065709 conn(0x7f86e00607c0 msgr2=0x7f86e0159c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.739+0000 7f86e51ef700 1 -- 172.21.15.3:0/2130065709 shutdown_connections 2023-03-03T06:26:08.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.739+0000 7f86e51ef700 1 -- 172.21.15.3:0/2130065709 wait complete. 2023-03-03T06:26:08.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-03T06:26:08.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.983+0000 7f4fe8bdf700 1 Processor -- start 2023-03-03T06:26:08.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.983+0000 7f4fe8bdf700 1 -- start start 2023-03-03T06:26:08.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.987+0000 7f4fe8bdf700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4fe415d6e0 0x7f4fe40cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.987+0000 7f4fe8bdf700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe40cf3b0 con 0x7f4fe4156ba0 2023-03-03T06:26:08.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.987+0000 7f4fe8bdf700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe4156f70 con 0x7f4fe40cfd80 2023-03-03T06:26:08.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.987+0000 7f4fe8bdf700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe40cd320 con 0x7f4fe415d6e0 2023-03-03T06:26:08.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.987+0000 7f4fe37fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4fe4156ba0 0x7f4fe4071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43384/0 (socket says 172.21.15.3:43384) 2023-03-03T06:26:08.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.987+0000 7f4fe37fe700 1 -- 172.21.15.3:0/3833756053 learned_addr learned my addr 172.21.15.3:0/3833756053 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:08.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.987+0000 7f4fe2ffd700 1 --2- 172.21.15.3:0/3833756053 >> v2:172.21.15.3:3300/0 conn(0x7f4fe415d6e0 0x7f4fe40cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.993 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.987+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346788067 0 0) 0x7f4fe40cf3b0 con 0x7f4fe4156ba0 2023-03-03T06:26:08.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.987+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fcc0035d0 con 0x7f4fe4156ba0 2023-03-03T06:26:08.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3022366911 0 0) 0x7f4fcc0035d0 con 0x7f4fe4156ba0 2023-03-03T06:26:08.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 >> v2:172.21.15.3:3300/0 conn(0x7f4fe415d6e0 msgr2=0x7f4fe40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:08.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 --2- 172.21.15.3:0/3833756053 >> v2:172.21.15.3:3300/0 conn(0x7f4fe415d6e0 0x7f4fe40cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 >> v1:172.21.15.32:6789/0 conn(0x7f4fe40cfd80 legacy=0x7f4fe40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe40cd600 con 0x7f4fe4156ba0 2023-03-03T06:26:08.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f4fd4001670 con 0x7f4fe4156ba0 2023-03-03T06:26:08.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 >> v1:172.21.15.3:6789/0 conn(0x7f4fe4156ba0 legacy=0x7f4fe4071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 --2- 172.21.15.3:0/3833756053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fcc003a20 0x7f4fcc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 --2- 172.21.15.3:0/3833756053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fcc004410 0x7f4fcc0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 --2- 172.21.15.3:0/3833756053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fcc0050d0 0x7f4fcc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:08.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40cd320 con 0x7f4fcc004410 2023-03-03T06:26:08.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc00c020 con 0x7f4fcc0050d0 2023-03-03T06:26:08.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0149f0 con 0x7f4fcc003a20 2023-03-03T06:26:08.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe2ffd700 1 --2- 172.21.15.3:0/3833756053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fcc004410 0x7f4fcc0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe37fe700 1 --2- 172.21.15.3:0/3833756053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fcc003a20 0x7f4fcc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe2ffd700 1 -- 172.21.15.3:0/3833756053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fcc003a20 msgr2=0x7f4fcc003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe2ffd700 1 --2- 172.21.15.3:0/3833756053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fcc003a20 0x7f4fcc003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe3fff700 1 --2- 172.21.15.3:0/3833756053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fcc0050d0 0x7f4fcc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe2ffd700 1 -- 172.21.15.3:0/3833756053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fcc0050d0 msgr2=0x7f4fcc00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe2ffd700 1 --2- 172.21.15.3:0/3833756053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fcc0050d0 0x7f4fcc00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe2ffd700 1 -- 172.21.15.3:0/3833756053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc0035d0 con 0x7f4fcc004410 2023-03-03T06:26:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe3fff700 1 --2- 172.21.15.3:0/3833756053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fcc0050d0 0x7f4fcc00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe3fff700 1 -- 172.21.15.3:0/3833756053 reap_dead start 2023-03-03T06:26:08.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe2ffd700 1 --2- 172.21.15.3:0/3833756053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fcc004410 0x7f4fcc0048d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009b70 tx=0x7f4fcc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548c1ba4a2a6885f server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:08.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd800e6c0 con 0x7f4fcc004410 2023-03-03T06:26:08.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd800e820 con 0x7f4fcc004410 2023-03-03T06:26:08.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe27fc700 1 -- 172.21.15.3:0/3833756053 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd800eb90 con 0x7f4fcc004410 2023-03-03T06:26:08.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/3833756053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fcc004410 msgr2=0x7f4fcc0048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:08.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe8bdf700 1 --2- 172.21.15.3:0/3833756053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fcc004410 0x7f4fcc0048d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009b70 tx=0x7f4fcc0058a0 comp rx=0 tx=0).stop 2023-03-03T06:26:08.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/3833756053 shutdown_connections 2023-03-03T06:26:08.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe8bdf700 1 --2- 172.21.15.3:0/3833756053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fcc004410 0x7f4fcc0048d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.991+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/3833756053 >> 172.21.15.3:0/3833756053 conn(0x7f4fe406e5c0 msgr2=0x7f4fe406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:09.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.995+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/3833756053 shutdown_connections 2023-03-03T06:26:09.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.995+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/3833756053 wait complete. 2023-03-03T06:26:09.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.995+0000 7f4fe8bdf700 1 Processor -- start 2023-03-03T06:26:09.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.995+0000 7f4fe8bdf700 1 -- start start 2023-03-03T06:26:09.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.995+0000 7f4fe8bdf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fcc004410 0x7f4fe4152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.995+0000 7f4fe8bdf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe4141740 0x7f4fe41524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.995+0000 7f4fe8bdf700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe40c8770 0x7f4fe40c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.995+0000 7f4fe8bdf700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40c9100 con 0x7f4fcc004410 2023-03-03T06:26:09.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.995+0000 7f4fe8bdf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4154030 con 0x7f4fe4141740 2023-03-03T06:26:09.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.995+0000 7f4fe8bdf700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4153420 con 0x7f4fe40c8770 2023-03-03T06:26:09.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fcc004410 0x7f4fe4152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fcc004410 0x7f4fe4152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59204/0 (socket says 172.21.15.3:59204) 2023-03-03T06:26:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe37fe700 1 -- 172.21.15.3:0/2813512922 learned_addr learned my addr 172.21.15.3:0/2813512922 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe3fff700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe40c8770 0x7f4fe40c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe2ffd700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe4141740 0x7f4fe41524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe37fe700 1 -- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe40c8770 msgr2=0x7f4fe40c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe37fe700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe40c8770 0x7f4fe40c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe37fe700 1 -- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe4141740 msgr2=0x7f4fe41524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe37fe700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe4141740 0x7f4fe41524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe37fe700 1 -- 172.21.15.3:0/2813512922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe40cef40 con 0x7f4fcc004410 2023-03-03T06:26:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe2ffd700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe4141740 0x7f4fe41524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe37fe700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fcc004410 0x7f4fe4152d20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400eaf0 tx=0x7f4fd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fcbfff700 1 -- 172.21.15.3:0/2813512922 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd400c7b0 con 0x7f4fcc004410 2023-03-03T06:26:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fcbfff700 1 -- 172.21.15.3:0/2813512922 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd400c930 con 0x7f4fcc004410 2023-03-03T06:26:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fcbfff700 1 -- 172.21.15.3:0/2813512922 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd400cce0 con 0x7f4fcc004410 2023-03-03T06:26:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/2813512922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe40cd8e0 con 0x7f4fcc004410 2023-03-03T06:26:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:08.999+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/2813512922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe40cf0d0 con 0x7f4fcc004410 2023-03-03T06:26:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.003+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/2813512922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe4073e90 con 0x7f4fcc004410 2023-03-03T06:26:09.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.003+0000 7f4fcbfff700 1 -- 172.21.15.3:0/2813512922 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4fd401d460 con 0x7f4fcc004410 2023-03-03T06:26:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.003+0000 7f4fcbfff700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4fd0064f10 0x7f4fd00673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.007+0000 7f4fcbfff700 1 -- 172.21.15.3:0/2813512922 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4fd4082420 con 0x7f4fcc004410 2023-03-03T06:26:09.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.007+0000 7f4fe2ffd700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4fd0064f10 0x7f4fd00673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.007+0000 7f4fe2ffd700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4fd0064f10 0x7f4fd00673c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80090b0 tx=0x7f4fd8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:09.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.011+0000 7f4fcbfff700 1 -- 172.21.15.3:0/2813512922 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4fd40500b0 con 0x7f4fcc004410 2023-03-03T06:26:09.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.187+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/2813512922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fe40b7350 con 0x7f4fcc004410 2023-03-03T06:26:09.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.187+0000 7f4fcbfff700 1 -- 172.21.15.3:0/2813512922 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fd40536d0 con 0x7f4fcc004410 2023-03-03T06:26:09.191 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-03-03T06:26:09.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.191+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4fd0064f10 msgr2=0x7f4fd00673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.191+0000 7f4fe8bdf700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4fd0064f10 0x7f4fd00673c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80090b0 tx=0x7f4fd8008030 comp rx=0 tx=0).stop 2023-03-03T06:26:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.191+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/2813512922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fcc004410 msgr2=0x7f4fe4152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.191+0000 7f4fe8bdf700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fcc004410 0x7f4fe4152d20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400eaf0 tx=0x7f4fd400eeb0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.191+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/2813512922 shutdown_connections 2023-03-03T06:26:09.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.191+0000 7f4fe8bdf700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4fd0064f10 0x7f4fd00673c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.191+0000 7f4fe8bdf700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4fe40c8770 0x7f4fe40c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.191+0000 7f4fe8bdf700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4fe4141740 0x7f4fe41524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.191+0000 7f4fe8bdf700 1 --2- 172.21.15.3:0/2813512922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4fcc004410 0x7f4fe4152d20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.191+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/2813512922 >> 172.21.15.3:0/2813512922 conn(0x7f4fe406e5c0 msgr2=0x7f4fe4155a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:09.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.191+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/2813512922 shutdown_connections 2023-03-03T06:26:09.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.191+0000 7f4fe8bdf700 1 -- 172.21.15.3:0/2813512922 wait complete. 2023-03-03T06:26:09.230 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-03T06:26:09.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.443+0000 7f1d7759e700 1 Processor -- start 2023-03-03T06:26:09.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.447+0000 7f1d7759e700 1 -- start start 2023-03-03T06:26:09.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.447+0000 7f1d7759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d7806a780 0x7f1d780cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.447+0000 7f1d7759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d780cb9a0 0x7f1d78158400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d7759e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7815c230 con 0x7f1d78158940 2023-03-03T06:26:09.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d7759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d780caac0 con 0x7f1d7806a780 2023-03-03T06:26:09.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d7759e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d780cd130 con 0x7f1d780cb9a0 2023-03-03T06:26:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d75d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d780cb9a0 0x7f1d78158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d75d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d780cb9a0 0x7f1d78158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:60640/0 (socket says 172.21.15.3:60640) 2023-03-03T06:26:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d75d9b700 1 -- 172.21.15.3:0/4089240790 learned_addr learned my addr 172.21.15.3:0/4089240790 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d7659c700 1 --2- 172.21.15.3:0/4089240790 >> v2:172.21.15.32:3300/0 conn(0x7f1d7806a780 0x7f1d780cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d7659c700 1 -- 172.21.15.3:0/4089240790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d780cb9a0 msgr2=0x7f1d78158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:09.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d7659c700 1 --2- 172.21.15.3:0/4089240790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d780cb9a0 0x7f1d78158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d7659c700 1 -- 172.21.15.3:0/4089240790 >> v1:172.21.15.3:6789/0 conn(0x7f1d78158940 legacy=0x7f1d780709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d7659c700 1 -- 172.21.15.3:0/4089240790 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d780cbf60 con 0x7f1d7806a780 2023-03-03T06:26:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d75d9b700 1 --2- 172.21.15.3:0/4089240790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d780cb9a0 0x7f1d78158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:09.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d7659c700 1 --2- 172.21.15.3:0/4089240790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7806a780 0x7f1d780cace0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800baf0 tx=0x7f1d6800be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753e1658a9637f96 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d7559a700 1 -- 172.21.15.3:0/4089240790 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800db70 con 0x7f1d7806a780 2023-03-03T06:26:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d7559a700 1 -- 172.21.15.3:0/4089240790 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6800dcd0 con 0x7f1d7806a780 2023-03-03T06:26:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.451+0000 7f1d7559a700 1 -- 172.21.15.3:0/4089240790 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68004030 con 0x7f1d7806a780 2023-03-03T06:26:09.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.455+0000 7f1d7759e700 1 -- 172.21.15.3:0/4089240790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7806a780 msgr2=0x7f1d780cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.455+0000 7f1d7759e700 1 --2- 172.21.15.3:0/4089240790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7806a780 0x7f1d780cace0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800baf0 tx=0x7f1d6800be00 comp rx=0 tx=0).stop 2023-03-03T06:26:09.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.455+0000 7f1d7759e700 1 -- 172.21.15.3:0/4089240790 shutdown_connections 2023-03-03T06:26:09.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.455+0000 7f1d7759e700 1 --2- 172.21.15.3:0/4089240790 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d780cb9a0 0x7f1d78158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.455+0000 7f1d7759e700 1 --2- 172.21.15.3:0/4089240790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d7806a780 0x7f1d780cace0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.455+0000 7f1d7759e700 1 -- 172.21.15.3:0/4089240790 >> 172.21.15.3:0/4089240790 conn(0x7f1d780607c0 msgr2=0x7f1d78060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:09.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.455+0000 7f1d7759e700 1 -- 172.21.15.3:0/4089240790 shutdown_connections 2023-03-03T06:26:09.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.455+0000 7f1d7759e700 1 -- 172.21.15.3:0/4089240790 wait complete. 2023-03-03T06:26:09.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d7759e700 1 Processor -- start 2023-03-03T06:26:09.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d7759e700 1 -- start start 2023-03-03T06:26:09.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d7759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7806a780 0x7f1d780b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d7759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d780cb9a0 0x7f1d780c17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d7759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d78158940 0x7f1d781513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d7759e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d78151900 con 0x7f1d78158940 2023-03-03T06:26:09.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d7759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d780c3ed0 con 0x7f1d780cb9a0 2023-03-03T06:26:09.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d7759e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d780c22a0 con 0x7f1d7806a780 2023-03-03T06:26:09.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d76d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d78158940 0x7f1d781513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d76d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d78158940 0x7f1d781513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59212/0 (socket says 172.21.15.3:59212) 2023-03-03T06:26:09.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d76d9d700 1 -- 172.21.15.3:0/3374420636 learned_addr learned my addr 172.21.15.3:0/3374420636 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:09.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d7659c700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7806a780 0x7f1d780b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d75d9b700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d780cb9a0 0x7f1d780c17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d76d9d700 1 -- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7806a780 msgr2=0x7f1d780b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d76d9d700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7806a780 0x7f1d780b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d76d9d700 1 -- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d780cb9a0 msgr2=0x7f1d780c17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.459+0000 7f1d76d9d700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d780cb9a0 0x7f1d780c17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.463+0000 7f1d76d9d700 1 -- 172.21.15.3:0/3374420636 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c009570 con 0x7f1d78158940 2023-03-03T06:26:09.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.463+0000 7f1d7659c700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7806a780 0x7f1d780b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:09.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.463+0000 7f1d75d9b700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d780cb9a0 0x7f1d780c17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:09.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.463+0000 7f1d76d9d700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d78158940 0x7f1d781513c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1d70002080 tx=0x7f1d7000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:09.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.463+0000 7f1d66ffd700 1 -- 172.21.15.3:0/3374420636 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70003e50 con 0x7f1d78158940 2023-03-03T06:26:09.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.463+0000 7f1d66ffd700 1 -- 172.21.15.3:0/3374420636 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7000a9f0 con 0x7f1d78158940 2023-03-03T06:26:09.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.463+0000 7f1d66ffd700 1 -- 172.21.15.3:0/3374420636 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7000ada0 con 0x7f1d78158940 2023-03-03T06:26:09.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.463+0000 7f1d7759e700 1 -- 172.21.15.3:0/3374420636 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6800b750 con 0x7f1d78158940 2023-03-03T06:26:09.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.463+0000 7f1d7759e700 1 -- 172.21.15.3:0/3374420636 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7815c7d0 con 0x7f1d78158940 2023-03-03T06:26:09.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.463+0000 7f1d7759e700 1 -- 172.21.15.3:0/3374420636 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7815c4f0 con 0x7f1d78158940 2023-03-03T06:26:09.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.467+0000 7f1d66ffd700 1 -- 172.21.15.3:0/3374420636 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1d70010070 con 0x7f1d78158940 2023-03-03T06:26:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.471+0000 7f1d66ffd700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d58064ee0 0x7f1d58067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.471+0000 7f1d66ffd700 1 -- 172.21.15.3:0/3374420636 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1d70082330 con 0x7f1d78158940 2023-03-03T06:26:09.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.471+0000 7f1d7659c700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d58064ee0 0x7f1d58067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.471+0000 7f1d7659c700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d58064ee0 0x7f1d58067390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800b2c0 tx=0x7f1d68013000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.475+0000 7f1d66ffd700 1 -- 172.21.15.3:0/3374420636 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1d700501a0 con 0x7f1d78158940 2023-03-03T06:26:09.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.651+0000 7f1d7759e700 1 -- 172.21.15.3:0/3374420636 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1d7815cab0 con 0x7f1d78158940 2023-03-03T06:26:09.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.655+0000 7f1d66ffd700 1 -- 172.21.15.3:0/3374420636 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1d700537c0 con 0x7f1d78158940 2023-03-03T06:26:09.658 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:09.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.655+0000 7f1d7759e700 1 -- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d58064ee0 msgr2=0x7f1d58067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.655+0000 7f1d7759e700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d58064ee0 0x7f1d58067390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800b2c0 tx=0x7f1d68013000 comp rx=0 tx=0).stop 2023-03-03T06:26:09.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.655+0000 7f1d7759e700 1 -- 172.21.15.3:0/3374420636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d78158940 msgr2=0x7f1d781513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.655+0000 7f1d7759e700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d78158940 0x7f1d781513c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1d70002080 tx=0x7f1d7000a340 comp rx=0 tx=0).stop 2023-03-03T06:26:09.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.655+0000 7f1d7759e700 1 -- 172.21.15.3:0/3374420636 shutdown_connections 2023-03-03T06:26:09.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.655+0000 7f1d7759e700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d58064ee0 0x7f1d58067390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.655+0000 7f1d7759e700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d78158940 0x7f1d781513c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.655+0000 7f1d7759e700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d780cb9a0 0x7f1d780c17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.655+0000 7f1d7759e700 1 --2- 172.21.15.3:0/3374420636 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d7806a780 0x7f1d780b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.655+0000 7f1d7759e700 1 -- 172.21.15.3:0/3374420636 >> 172.21.15.3:0/3374420636 conn(0x7f1d780607c0 msgr2=0x7f1d780c95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:09.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.659+0000 7f1d7759e700 1 -- 172.21.15.3:0/3374420636 shutdown_connections 2023-03-03T06:26:09.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.659+0000 7f1d7759e700 1 -- 172.21.15.3:0/3374420636 wait complete. 2023-03-03T06:26:09.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.895+0000 7f94ad3fc700 1 Processor -- start 2023-03-03T06:26:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.895+0000 7f94ad3fc700 1 -- start start 2023-03-03T06:26:09.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94ad3fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94a80b2370 0x7f94a8160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94ad3fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8145e20 0x7f94a80ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94ad3fc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a80ce1c0 con 0x7f94a80cad70 2023-03-03T06:26:09.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94ad3fc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a80cb210 con 0x7f94a80b2370 2023-03-03T06:26:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94ad3fc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a8160f30 con 0x7f94a8145e20 2023-03-03T06:26:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94a7fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f94a80cad70 0x7f94a80ccbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43400/0 (socket says 172.21.15.3:43400) 2023-03-03T06:26:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94a7fff700 1 -- 172.21.15.3:0/3808810711 learned_addr learned my addr 172.21.15.3:0/3808810711 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94a67fc700 1 --2- 172.21.15.3:0/3808810711 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8145e20 0x7f94a80ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94a6ffd700 1 --2- 172.21.15.3:0/3808810711 >> v2:172.21.15.32:3300/0 conn(0x7f94a80b2370 0x7f94a8160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94a5ffb700 1 -- 172.21.15.3:0/3808810711 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349079246 0 0) 0x7f94a80ce1c0 con 0x7f94a80cad70 2023-03-03T06:26:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94a5ffb700 1 -- 172.21.15.3:0/3808810711 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94900035d0 con 0x7f94a80cad70 2023-03-03T06:26:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94a6ffd700 1 -- 172.21.15.3:0/3808810711 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8145e20 msgr2=0x7f94a80ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94a6ffd700 1 --2- 172.21.15.3:0/3808810711 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8145e20 0x7f94a80ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94a6ffd700 1 -- 172.21.15.3:0/3808810711 >> v1:172.21.15.3:6789/0 conn(0x7f94a80cad70 legacy=0x7f94a80ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.899+0000 7f94a6ffd700 1 -- 172.21.15.3:0/3808810711 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a8160970 con 0x7f94a80b2370 2023-03-03T06:26:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.903+0000 7f94a6ffd700 1 --2- 172.21.15.3:0/3808810711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a80b2370 0x7f94a8160000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9498009850 tx=0x7f9498009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fb384ba86d0713a server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.903+0000 7f94a5ffb700 1 -- 172.21.15.3:0/3808810711 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498011830 con 0x7f94a80b2370 2023-03-03T06:26:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.903+0000 7f94a5ffb700 1 -- 172.21.15.3:0/3808810711 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9498011990 con 0x7f94a80b2370 2023-03-03T06:26:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.903+0000 7f94a5ffb700 1 -- 172.21.15.3:0/3808810711 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498011d40 con 0x7f94a80b2370 2023-03-03T06:26:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.903+0000 7f94ad3fc700 1 -- 172.21.15.3:0/3808810711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a80b2370 msgr2=0x7f94a8160000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.903+0000 7f94ad3fc700 1 --2- 172.21.15.3:0/3808810711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a80b2370 0x7f94a8160000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9498009850 tx=0x7f9498009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.903+0000 7f94ad3fc700 1 -- 172.21.15.3:0/3808810711 shutdown_connections 2023-03-03T06:26:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.903+0000 7f94ad3fc700 1 --2- 172.21.15.3:0/3808810711 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8145e20 0x7f94a80ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.903+0000 7f94ad3fc700 1 --2- 172.21.15.3:0/3808810711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a80b2370 0x7f94a8160000 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.903+0000 7f94ad3fc700 1 -- 172.21.15.3:0/3808810711 >> 172.21.15.3:0/3808810711 conn(0x7f94a806b580 msgr2=0x7f94a806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.903+0000 7f94ad3fc700 1 -- 172.21.15.3:0/3808810711 shutdown_connections 2023-03-03T06:26:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.903+0000 7f94ad3fc700 1 -- 172.21.15.3:0/3808810711 wait complete. 2023-03-03T06:26:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.907+0000 7f94ad3fc700 1 Processor -- start 2023-03-03T06:26:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.907+0000 7f94ad3fc700 1 -- start start 2023-03-03T06:26:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.907+0000 7f94ad3fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a80b2370 0x7f94a80c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94ad3fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a80cad70 0x7f94a8070120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94ad3fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8145e20 0x7f94a8070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94ad3fc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a814df20 con 0x7f94a80b2370 2023-03-03T06:26:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94ad3fc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80b5540 con 0x7f94a80cad70 2023-03-03T06:26:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94ad3fc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a814d310 con 0x7f94a8145e20 2023-03-03T06:26:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94a6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a80b2370 0x7f94a80c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94a6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a80b2370 0x7f94a80c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59222/0 (socket says 172.21.15.3:59222) 2023-03-03T06:26:09.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94a6ffd700 1 -- 172.21.15.3:0/377551973 learned_addr learned my addr 172.21.15.3:0/377551973 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94a67fc700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a80cad70 0x7f94a8070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94a7fff700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8145e20 0x7f94a8070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94a6ffd700 1 -- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8145e20 msgr2=0x7f94a8070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94a6ffd700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8145e20 0x7f94a8070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94a6ffd700 1 -- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a80cad70 msgr2=0x7f94a8070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94a6ffd700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a80cad70 0x7f94a8070120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94a6ffd700 1 -- 172.21.15.3:0/377551973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94980094b0 con 0x7f94a80b2370 2023-03-03T06:26:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94a6ffd700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a80b2370 0x7f94a80c6170 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9498014040 tx=0x7f9498004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f948f7fe700 1 -- 172.21.15.3:0/377551973 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498004510 con 0x7f94a80b2370 2023-03-03T06:26:09.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f948f7fe700 1 -- 172.21.15.3:0/377551973 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949801e070 con 0x7f94a80b2370 2023-03-03T06:26:09.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94ad3fc700 1 -- 172.21.15.3:0/377551973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a80cde50 con 0x7f94a80b2370 2023-03-03T06:26:09.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f948f7fe700 1 -- 172.21.15.3:0/377551973 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498003900 con 0x7f94a80b2370 2023-03-03T06:26:09.918 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.911+0000 7f94ad3fc700 1 -- 172.21.15.3:0/377551973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a8160c50 con 0x7f94a80b2370 2023-03-03T06:26:09.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.915+0000 7f94ad3fc700 1 -- 172.21.15.3:0/377551973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a80b2740 con 0x7f94a80b2370 2023-03-03T06:26:09.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.915+0000 7f948f7fe700 1 -- 172.21.15.3:0/377551973 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f949800f050 con 0x7f94a80b2370 2023-03-03T06:26:09.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.919+0000 7f948f7fe700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9494064f60 0x7f9494067410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.919+0000 7f948f7fe700 1 -- 172.21.15.3:0/377551973 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9498020070 con 0x7f94a80b2370 2023-03-03T06:26:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.919+0000 7f94a67fc700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9494064f60 0x7f9494067410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:09.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.923+0000 7f94a67fc700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9494064f60 0x7f9494067410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f94a80b5860 tx=0x7f949c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:09.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:09.923+0000 7f948f7fe700 1 -- 172.21.15.3:0/377551973 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f94980512e0 con 0x7f94a80b2370 2023-03-03T06:26:10.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.111+0000 7f94ad3fc700 1 -- 172.21.15.3:0/377551973 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94a804e180 con 0x7f94a80b2370 2023-03-03T06:26:10.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.115+0000 7f948f7fe700 1 -- 172.21.15.3:0/377551973 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f9498006ce0 con 0x7f94a80b2370 2023-03-03T06:26:10.119 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:10.119 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:53.877945+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:10.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.115+0000 7f94ad3fc700 1 -- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9494064f60 msgr2=0x7f9494067410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:10.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.115+0000 7f94ad3fc700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9494064f60 0x7f9494067410 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f94a80b5860 tx=0x7f949c009240 comp rx=0 tx=0).stop 2023-03-03T06:26:10.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.115+0000 7f94ad3fc700 1 -- 172.21.15.3:0/377551973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a80b2370 msgr2=0x7f94a80c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:10.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.115+0000 7f94ad3fc700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a80b2370 0x7f94a80c6170 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9498014040 tx=0x7f9498004770 comp rx=0 tx=0).stop 2023-03-03T06:26:10.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.115+0000 7f94ad3fc700 1 -- 172.21.15.3:0/377551973 shutdown_connections 2023-03-03T06:26:10.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.115+0000 7f94ad3fc700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9494064f60 0x7f9494067410 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.115+0000 7f94ad3fc700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a8145e20 0x7f94a8070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.115+0000 7f94ad3fc700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a80cad70 0x7f94a8070120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.115+0000 7f94ad3fc700 1 --2- 172.21.15.3:0/377551973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a80b2370 0x7f94a80c6170 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.115+0000 7f94ad3fc700 1 -- 172.21.15.3:0/377551973 >> 172.21.15.3:0/377551973 conn(0x7f94a806b580 msgr2=0x7f94a815ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:10.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.119+0000 7f94ad3fc700 1 -- 172.21.15.3:0/377551973 shutdown_connections 2023-03-03T06:26:10.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.119+0000 7f94ad3fc700 1 -- 172.21.15.3:0/377551973 wait complete. 2023-03-03T06:26:10.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-03T06:25:11.468534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-03T06:25:12.813283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-03T06:25:16.382800+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-03T06:26:10.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:26:10.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.375+0000 7f844359e700 1 Processor -- start 2023-03-03T06:26:10.381 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.375+0000 7f844359e700 1 -- start start 2023-03-03T06:26:10.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.379+0000 7f844359e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f844415faf0 0x7f84440b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.379+0000 7f844359e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f84440b4d90 0x7f84440b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.379+0000 7f844359e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84440cd2f0 con 0x7f84440cd7d0 2023-03-03T06:26:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.379+0000 7f844359e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84441609d0 con 0x7f844415faf0 2023-03-03T06:26:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.379+0000 7f844359e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84440cf520 con 0x7f84440b4d90 2023-03-03T06:26:10.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.379+0000 7f8441d9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f84440cd7d0 0x7f84440b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52364/0 (socket says 172.21.15.3:52364) 2023-03-03T06:26:10.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.379+0000 7f8441d9b700 1 -- 172.21.15.3:0/3255119118 learned_addr learned my addr 172.21.15.3:0/3255119118 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:10.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.379+0000 7f844259c700 1 --2- 172.21.15.3:0/3255119118 >> v2:172.21.15.32:3300/0 conn(0x7f844415faf0 0x7f84440b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:10.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.379+0000 7f8442d9d700 1 --2- 172.21.15.3:0/3255119118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f84440b4d90 0x7f84440b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:10.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.383+0000 7f844159a700 1 -- 172.21.15.3:0/3255119118 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325485805 0 0) 0x7f84440cd2f0 con 0x7f84440cd7d0 2023-03-03T06:26:10.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.383+0000 7f844159a700 1 -- 172.21.15.3:0/3255119118 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f842c0035d0 con 0x7f84440cd7d0 2023-03-03T06:26:10.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.383+0000 7f844259c700 1 -- 172.21.15.3:0/3255119118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f84440b4d90 msgr2=0x7f84440b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:10.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.383+0000 7f844259c700 1 --2- 172.21.15.3:0/3255119118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f84440b4d90 0x7f84440b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.383+0000 7f844259c700 1 -- 172.21.15.3:0/3255119118 >> v1:172.21.15.3:6789/0 conn(0x7f84440cd7d0 legacy=0x7f84440b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:10.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.383+0000 7f844159a700 1 -- 172.21.15.3:0/3255119118 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2152122579 0 0) 0x7f842c0035d0 con 0x7f84440cd7d0 2023-03-03T06:26:10.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.383+0000 7f844259c700 1 -- 172.21.15.3:0/3255119118 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84440cf240 con 0x7f844415faf0 2023-03-03T06:26:10.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.383+0000 7f844259c700 1 --2- 172.21.15.3:0/3255119118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844415faf0 0x7f84440b1ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8434009850 tx=0x7f8434009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f769057342297dce server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:10.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.383+0000 7f844159a700 1 -- 172.21.15.3:0/3255119118 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438001670 con 0x7f844415faf0 2023-03-03T06:26:10.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.383+0000 7f844159a700 1 -- 172.21.15.3:0/3255119118 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8434004030 con 0x7f844415faf0 2023-03-03T06:26:10.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.383+0000 7f844159a700 1 -- 172.21.15.3:0/3255119118 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340043e0 con 0x7f844415faf0 2023-03-03T06:26:10.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.387+0000 7f844359e700 1 -- 172.21.15.3:0/3255119118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844415faf0 msgr2=0x7f84440b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:10.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.387+0000 7f844359e700 1 --2- 172.21.15.3:0/3255119118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844415faf0 0x7f84440b1ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8434009850 tx=0x7f8434009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:10.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.387+0000 7f844359e700 1 -- 172.21.15.3:0/3255119118 shutdown_connections 2023-03-03T06:26:10.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.387+0000 7f844359e700 1 --2- 172.21.15.3:0/3255119118 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f84440b4d90 0x7f84440b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.387+0000 7f844359e700 1 --2- 172.21.15.3:0/3255119118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f844415faf0 0x7f84440b1ac0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.387+0000 7f844359e700 1 -- 172.21.15.3:0/3255119118 >> 172.21.15.3:0/3255119118 conn(0x7f84440cf680 msgr2=0x7f84440cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:10.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.387+0000 7f844359e700 1 -- 172.21.15.3:0/3255119118 shutdown_connections 2023-03-03T06:26:10.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.387+0000 7f844359e700 1 -- 172.21.15.3:0/3255119118 wait complete. 2023-03-03T06:26:10.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.391+0000 7f844359e700 1 Processor -- start 2023-03-03T06:26:10.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.391+0000 7f844359e700 1 -- start start 2023-03-03T06:26:10.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.391+0000 7f844359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84440b4d90 0x7f844414a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:10.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.391+0000 7f844359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84440cd7d0 0x7f84441f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:10.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.391+0000 7f844359e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f844415faf0 0x7f84440c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:10.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.391+0000 7f844359e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844414a5c0 con 0x7f84440cd7d0 2023-03-03T06:26:10.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.391+0000 7f844359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84441f6550 con 0x7f84440b4d90 2023-03-03T06:26:10.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.391+0000 7f844359e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84441f5f90 con 0x7f844415faf0 2023-03-03T06:26:10.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.391+0000 7f8441d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84440cd7d0 0x7f84441f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:10.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.391+0000 7f8441d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84440cd7d0 0x7f84441f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44788/0 (socket says 172.21.15.3:44788) 2023-03-03T06:26:10.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.391+0000 7f8441d9b700 1 -- 172.21.15.3:0/1444797833 learned_addr learned my addr 172.21.15.3:0/1444797833 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:10.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.391+0000 7f844259c700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84440b4d90 0x7f844414a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:10.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.395+0000 7f8442d9d700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f844415faf0 0x7f84440c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:10.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.395+0000 7f8441d9b700 1 -- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f844415faf0 msgr2=0x7f84440c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:10.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.395+0000 7f8441d9b700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f844415faf0 0x7f84440c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.395+0000 7f8441d9b700 1 -- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84440b4d90 msgr2=0x7f844414a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:10.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.395+0000 7f8441d9b700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84440b4d90 0x7f844414a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.395+0000 7f8441d9b700 1 -- 172.21.15.3:0/1444797833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84340094b0 con 0x7f84440cd7d0 2023-03-03T06:26:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.395+0000 7f8441d9b700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84440cd7d0 0x7f84441f5620 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f843800c730 tx=0x7f843800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:10.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.395+0000 7f8432ffd700 1 -- 172.21.15.3:0/1444797833 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438004ca0 con 0x7f84440cd7d0 2023-03-03T06:26:10.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.395+0000 7f8432ffd700 1 -- 172.21.15.3:0/1444797833 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8438004e00 con 0x7f84440cd7d0 2023-03-03T06:26:10.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.395+0000 7f844359e700 1 -- 172.21.15.3:0/1444797833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8444160ea0 con 0x7f84440cd7d0 2023-03-03T06:26:10.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.395+0000 7f8432ffd700 1 -- 172.21.15.3:0/1444797833 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438007310 con 0x7f84440cd7d0 2023-03-03T06:26:10.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.395+0000 7f844359e700 1 -- 172.21.15.3:0/1444797833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8444160bc0 con 0x7f84440cd7d0 2023-03-03T06:26:10.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.399+0000 7f844359e700 1 -- 172.21.15.3:0/1444797833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84440ceef0 con 0x7f84440cd7d0 2023-03-03T06:26:10.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.399+0000 7f8432ffd700 1 -- 172.21.15.3:0/1444797833 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f843801a020 con 0x7f84440cd7d0 2023-03-03T06:26:10.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.403+0000 7f8432ffd700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8424064ee0 0x7f8424067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:10.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.403+0000 7f8432ffd700 1 -- 172.21.15.3:0/1444797833 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8438081e70 con 0x7f84440cd7d0 2023-03-03T06:26:10.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.403+0000 7f844259c700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8424064ee0 0x7f8424067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:10.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.407+0000 7f844259c700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8424064ee0 0x7f8424067390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f84340090b0 tx=0x7f8434008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:10.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.411+0000 7f8432ffd700 1 -- 172.21.15.3:0/1444797833 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f843804fcb0 con 0x7f84440cd7d0 2023-03-03T06:26:10.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.587+0000 7f844359e700 1 -- 172.21.15.3:0/1444797833 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84440720f0 con 0x7f84440cd7d0 2023-03-03T06:26:10.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.587+0000 7f8432ffd700 1 -- 172.21.15.3:0/1444797833 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84380532d0 con 0x7f84440cd7d0 2023-03-03T06:26:10.592 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:26:10.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.591+0000 7f844359e700 1 -- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8424064ee0 msgr2=0x7f8424067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:10.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.591+0000 7f844359e700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8424064ee0 0x7f8424067390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f84340090b0 tx=0x7f8434008030 comp rx=0 tx=0).stop 2023-03-03T06:26:10.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.591+0000 7f844359e700 1 -- 172.21.15.3:0/1444797833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84440cd7d0 msgr2=0x7f84441f5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:10.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.591+0000 7f844359e700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84440cd7d0 0x7f84441f5620 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f843800c730 tx=0x7f843800ca40 comp rx=0 tx=0).stop 2023-03-03T06:26:10.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.591+0000 7f844359e700 1 -- 172.21.15.3:0/1444797833 shutdown_connections 2023-03-03T06:26:10.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.591+0000 7f844359e700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8424064ee0 0x7f8424067390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.591+0000 7f844359e700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f844415faf0 0x7f84440c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.591+0000 7f844359e700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84440cd7d0 0x7f84441f5620 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.591+0000 7f844359e700 1 --2- 172.21.15.3:0/1444797833 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84440b4d90 0x7f844414a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.591+0000 7f844359e700 1 -- 172.21.15.3:0/1444797833 >> 172.21.15.3:0/1444797833 conn(0x7f84440cf680 msgr2=0x7f84440b7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:10.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.591+0000 7f844359e700 1 -- 172.21.15.3:0/1444797833 shutdown_connections 2023-03-03T06:26:10.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.591+0000 7f844359e700 1 -- 172.21.15.3:0/1444797833 wait complete. 2023-03-03T06:26:10.624 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:26:10.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.831+0000 7fc26f3f9700 1 Processor -- start 2023-03-03T06:26:10.837 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.831+0000 7fc26f3f9700 1 -- start start 2023-03-03T06:26:10.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.835+0000 7fc26f3f9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc26806c590 0x7fc2680b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:10.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.835+0000 7fc26f3f9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc2680b0e00 0x7fc2680c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:10.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.835+0000 7fc26f3f9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2680b1260 con 0x7fc2681591d0 2023-03-03T06:26:10.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.835+0000 7fc26f3f9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc26806d6d0 con 0x7fc2680b0e00 2023-03-03T06:26:10.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.835+0000 7fc26f3f9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2680cf6b0 con 0x7fc26806c590 2023-03-03T06:26:10.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26e3f7700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc2681591d0 0x7fc2680cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52366/0 (socket says 172.21.15.3:52366) 2023-03-03T06:26:10.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26e3f7700 1 -- 172.21.15.3:0/1932891187 learned_addr learned my addr 172.21.15.3:0/1932891187 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:10.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26dbf6700 1 --2- 172.21.15.3:0/1932891187 >> v2:172.21.15.32:3300/0 conn(0x7fc26806c590 0x7fc2680b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:10.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26ebf8700 1 --2- 172.21.15.3:0/1932891187 >> v2:172.21.15.3:3300/0 conn(0x7fc2680b0e00 0x7fc2680c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:10.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26d3f5700 1 -- 172.21.15.3:0/1932891187 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607654546 0 0) 0x7fc2680b1260 con 0x7fc2681591d0 2023-03-03T06:26:10.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26d3f5700 1 -- 172.21.15.3:0/1932891187 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2540035d0 con 0x7fc2681591d0 2023-03-03T06:26:10.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26ebf8700 1 -- 172.21.15.3:0/1932891187 >> v2:172.21.15.32:3300/0 conn(0x7fc26806c590 msgr2=0x7fc2680b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:10.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26ebf8700 1 --2- 172.21.15.3:0/1932891187 >> v2:172.21.15.32:3300/0 conn(0x7fc26806c590 0x7fc2680b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26ebf8700 1 -- 172.21.15.3:0/1932891187 >> v1:172.21.15.3:6789/0 conn(0x7fc2681591d0 legacy=0x7fc2680cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:10.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26ebf8700 1 -- 172.21.15.3:0/1932891187 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2681595a0 con 0x7fc2680b0e00 2023-03-03T06:26:10.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26ebf8700 1 --2- 172.21.15.3:0/1932891187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2680b0e00 0x7fc2680c9a10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc2680cc0c0 tx=0x7fc26000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dab804f3651e15 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:10.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26d3f5700 1 -- 172.21.15.3:0/1932891187 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26000e070 con 0x7fc2680b0e00 2023-03-03T06:26:10.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26d3f5700 1 -- 172.21.15.3:0/1932891187 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26000bb20 con 0x7fc2680b0e00 2023-03-03T06:26:10.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.839+0000 7fc26d3f5700 1 -- 172.21.15.3:0/1932891187 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260012430 con 0x7fc2680b0e00 2023-03-03T06:26:10.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.843+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1932891187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2680b0e00 msgr2=0x7fc2680c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.843+0000 7fc26f3f9700 1 --2- 172.21.15.3:0/1932891187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2680b0e00 0x7fc2680c9a10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc2680cc0c0 tx=0x7fc26000b330 comp rx=0 tx=0).stop 2023-03-03T06:26:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.843+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1932891187 shutdown_connections 2023-03-03T06:26:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.843+0000 7fc26f3f9700 1 --2- 172.21.15.3:0/1932891187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2680b0e00 0x7fc2680c9a10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.843+0000 7fc26f3f9700 1 --2- 172.21.15.3:0/1932891187 >> v2:172.21.15.32:3300/0 conn(0x7fc26806c590 0x7fc2680b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.843+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1932891187 >> 172.21.15.3:0/1932891187 conn(0x7fc2680607c0 msgr2=0x7fc268060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.843+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1932891187 shutdown_connections 2023-03-03T06:26:10.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.843+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1932891187 wait complete. 2023-03-03T06:26:10.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.843+0000 7fc26f3f9700 1 Processor -- start 2023-03-03T06:26:10.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.843+0000 7fc26f3f9700 1 -- start start 2023-03-03T06:26:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26f3f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc26806c590 0x7fc2680c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26f3f9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2680b0e00 0x7fc2680c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26f3f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681591d0 0x7fc2681505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:10.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26f3f9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2680cbd90 con 0x7fc2680b0e00 2023-03-03T06:26:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26f3f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc268151300 con 0x7fc2681591d0 2023-03-03T06:26:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26f3f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2680c5880 con 0x7fc26806c590 2023-03-03T06:26:10.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26dbf6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2680b0e00 0x7fc2680c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26dbf6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2680b0e00 0x7fc2680c21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44816/0 (socket says 172.21.15.3:44816) 2023-03-03T06:26:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26dbf6700 1 -- 172.21.15.3:0/1827044952 learned_addr learned my addr 172.21.15.3:0/1827044952 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26e3f7700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc26806c590 0x7fc2680c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26ebf8700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681591d0 0x7fc2681505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26dbf6700 1 -- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc26806c590 msgr2=0x7fc2680c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26dbf6700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc26806c590 0x7fc2680c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26dbf6700 1 -- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681591d0 msgr2=0x7fc2681505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26dbf6700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681591d0 0x7fc2681505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26dbf6700 1 -- 172.21.15.3:0/1827044952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc260009a70 con 0x7fc2680b0e00 2023-03-03T06:26:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26e3f7700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc26806c590 0x7fc2680c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26ebf8700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681591d0 0x7fc2681505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:10.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26dbf6700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2680b0e00 0x7fc2680c21e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc25800e9a0 tx=0x7fc25800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc25effd700 1 -- 172.21.15.3:0/1827044952 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258009720 con 0x7fc2680b0e00 2023-03-03T06:26:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc25effd700 1 -- 172.21.15.3:0/1827044952 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25800c850 con 0x7fc2680b0e00 2023-03-03T06:26:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc25effd700 1 -- 172.21.15.3:0/1827044952 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25800cc00 con 0x7fc2680b0e00 2023-03-03T06:26:10.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1827044952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26806cd30 con 0x7fc2680b0e00 2023-03-03T06:26:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.847+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1827044952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26806ca50 con 0x7fc2680b0e00 2023-03-03T06:26:10.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.851+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1827044952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26806d010 con 0x7fc2680b0e00 2023-03-03T06:26:10.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.851+0000 7fc25effd700 1 -- 172.21.15.3:0/1827044952 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc258019030 con 0x7fc2680b0e00 2023-03-03T06:26:10.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.855+0000 7fc25effd700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc24c064ee0 0x7fc24c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:10.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.855+0000 7fc25effd700 1 -- 172.21.15.3:0/1827044952 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc2580821c0 con 0x7fc2680b0e00 2023-03-03T06:26:10.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.855+0000 7fc26e3f7700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc24c064ee0 0x7fc24c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:10.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.855+0000 7fc26e3f7700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc24c064ee0 0x7fc24c067390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc2640011e0 tx=0x7fc264009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:10.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:10.859+0000 7fc25effd700 1 -- 172.21.15.3:0/1827044952 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc258050030 con 0x7fc2680b0e00 2023-03-03T06:26:11.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.031+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1827044952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc2680cba70 con 0x7fc2680b0e00 2023-03-03T06:26:11.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.035+0000 7fc25effd700 1 -- 172.21.15.3:0/1827044952 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc258053650 con 0x7fc2680b0e00 2023-03-03T06:26:11.038 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:26:11.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.039+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc24c064ee0 msgr2=0x7fc24c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.039+0000 7fc26f3f9700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc24c064ee0 0x7fc24c067390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc2640011e0 tx=0x7fc264009040 comp rx=0 tx=0).stop 2023-03-03T06:26:11.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.039+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1827044952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2680b0e00 msgr2=0x7fc2680c21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.039+0000 7fc26f3f9700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2680b0e00 0x7fc2680c21e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc25800e9a0 tx=0x7fc25800ed60 comp rx=0 tx=0).stop 2023-03-03T06:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.039+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1827044952 shutdown_connections 2023-03-03T06:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.039+0000 7fc26f3f9700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc24c064ee0 0x7fc24c067390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.039+0000 7fc26f3f9700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2681591d0 0x7fc2681505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.039+0000 7fc26f3f9700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2680b0e00 0x7fc2680c21e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.039+0000 7fc26f3f9700 1 --2- 172.21.15.3:0/1827044952 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc26806c590 0x7fc2680c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.039+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1827044952 >> 172.21.15.3:0/1827044952 conn(0x7fc2680607c0 msgr2=0x7fc2680709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:11.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.039+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1827044952 shutdown_connections 2023-03-03T06:26:11.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.039+0000 7fc26f3f9700 1 -- 172.21.15.3:0/1827044952 wait complete. 2023-03-03T06:26:11.077 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-03T06:26:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.283+0000 7f1c2b445700 1 Processor -- start 2023-03-03T06:26:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.283+0000 7f1c2b445700 1 -- start start 2023-03-03T06:26:11.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.287+0000 7f1c2b445700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c2406a780 0x7f1c240cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:11.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.287+0000 7f1c2b445700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c240cb9a0 0x7f1c240709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:11.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.287+0000 7f1c2b445700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c240cf5f0 con 0x7f1c24070ef0 2023-03-03T06:26:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.287+0000 7f1c2b445700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c240caac0 con 0x7f1c2406a780 2023-03-03T06:26:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.287+0000 7f1c2b445700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24071370 con 0x7f1c240cb9a0 2023-03-03T06:26:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.287+0000 7f1c2ac44700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1c24070ef0 0x7f1c2406c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52382/0 (socket says 172.21.15.3:52382) 2023-03-03T06:26:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.287+0000 7f1c2ac44700 1 -- 172.21.15.3:0/1022799872 learned_addr learned my addr 172.21.15.3:0/1022799872 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.287+0000 7f1c2a443700 1 --2- 172.21.15.3:0/1022799872 >> v2:172.21.15.3:3300/0 conn(0x7f1c2406a780 0x7f1c240cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.287+0000 7f1c29c42700 1 --2- 172.21.15.3:0/1022799872 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c240cb9a0 0x7f1c240709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:11.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.291+0000 7f1c29441700 1 -- 172.21.15.3:0/1022799872 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303124774 0 0) 0x7f1c240cf5f0 con 0x7f1c24070ef0 2023-03-03T06:26:11.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.291+0000 7f1c29441700 1 -- 172.21.15.3:0/1022799872 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c100035d0 con 0x7f1c24070ef0 2023-03-03T06:26:11.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.291+0000 7f1c2a443700 1 -- 172.21.15.3:0/1022799872 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c240cb9a0 msgr2=0x7f1c240709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:11.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.291+0000 7f1c2a443700 1 --2- 172.21.15.3:0/1022799872 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c240cb9a0 0x7f1c240709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.291+0000 7f1c2a443700 1 -- 172.21.15.3:0/1022799872 >> v1:172.21.15.3:6789/0 conn(0x7f1c24070ef0 legacy=0x7f1c2406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.291+0000 7f1c29441700 1 -- 172.21.15.3:0/1022799872 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2553515691 0 0) 0x7f1c100035d0 con 0x7f1c24070ef0 2023-03-03T06:26:11.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.291+0000 7f1c2a443700 1 -- 172.21.15.3:0/1022799872 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2415c230 con 0x7f1c2406a780 2023-03-03T06:26:11.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.291+0000 7f1c2a443700 1 --2- 172.21.15.3:0/1022799872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406a780 0x7f1c240cace0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009850 tx=0x7f1c20009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ade329a9839810 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:11.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.291+0000 7f1c29441700 1 -- 172.21.15.3:0/1022799872 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0019f0 con 0x7f1c2406a780 2023-03-03T06:26:11.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.291+0000 7f1c29441700 1 -- 172.21.15.3:0/1022799872 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c20004030 con 0x7f1c2406a780 2023-03-03T06:26:11.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.291+0000 7f1c29441700 1 -- 172.21.15.3:0/1022799872 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200043e0 con 0x7f1c2406a780 2023-03-03T06:26:11.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.291+0000 7f1c2b445700 1 -- 172.21.15.3:0/1022799872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406a780 msgr2=0x7f1c240cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.291+0000 7f1c2b445700 1 --2- 172.21.15.3:0/1022799872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406a780 0x7f1c240cace0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009850 tx=0x7f1c20009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.295+0000 7f1c2b445700 1 -- 172.21.15.3:0/1022799872 shutdown_connections 2023-03-03T06:26:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.295+0000 7f1c2b445700 1 --2- 172.21.15.3:0/1022799872 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c240cb9a0 0x7f1c240709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.295+0000 7f1c2b445700 1 --2- 172.21.15.3:0/1022799872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c2406a780 0x7f1c240cace0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.295+0000 7f1c2b445700 1 -- 172.21.15.3:0/1022799872 >> 172.21.15.3:0/1022799872 conn(0x7f1c240607c0 msgr2=0x7f1c24060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.295+0000 7f1c2b445700 1 -- 172.21.15.3:0/1022799872 shutdown_connections 2023-03-03T06:26:11.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.295+0000 7f1c2b445700 1 -- 172.21.15.3:0/1022799872 wait complete. 2023-03-03T06:26:11.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.295+0000 7f1c2b445700 1 Processor -- start 2023-03-03T06:26:11.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c2b445700 1 -- start start 2023-03-03T06:26:11.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c2b445700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2406a780 0x7f1c2414e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:11.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c2b445700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c24070ef0 0x7f1c24153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:11.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c2b445700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c240cb9a0 0x7f1c24153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:11.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c2b445700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c241f4e80 con 0x7f1c24070ef0 2023-03-03T06:26:11.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c2b445700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c241f5160 con 0x7f1c2406a780 2023-03-03T06:26:11.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c2b445700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c241f5440 con 0x7f1c240cb9a0 2023-03-03T06:26:11.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c29c42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c24070ef0 0x7f1c24153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:11.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c29c42700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c24070ef0 0x7f1c24153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44842/0 (socket says 172.21.15.3:44842) 2023-03-03T06:26:11.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c29c42700 1 -- 172.21.15.3:0/49716064 learned_addr learned my addr 172.21.15.3:0/49716064 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:11.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c2a443700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2406a780 0x7f1c2414e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:11.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c2ac44700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c240cb9a0 0x7f1c24153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:11.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c29c42700 1 -- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c240cb9a0 msgr2=0x7f1c24153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c29c42700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c240cb9a0 0x7f1c24153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c29c42700 1 -- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2406a780 msgr2=0x7f1c2414e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c29c42700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2406a780 0x7f1c2414e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c29c42700 1 -- 172.21.15.3:0/49716064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200094b0 con 0x7f1c24070ef0 2023-03-03T06:26:11.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c2ac44700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c240cb9a0 0x7f1c24153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:11.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c2a443700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2406a780 0x7f1c2414e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:11.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.299+0000 7f1c29c42700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c24070ef0 0x7f1c24153520 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400e9d0 tx=0x7f1c1400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:11.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.303+0000 7f1c1affd700 1 -- 172.21.15.3:0/49716064 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14009720 con 0x7f1c24070ef0 2023-03-03T06:26:11.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.303+0000 7f1c1affd700 1 -- 172.21.15.3:0/49716064 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1400c850 con 0x7f1c24070ef0 2023-03-03T06:26:11.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.303+0000 7f1c1affd700 1 -- 172.21.15.3:0/49716064 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1400cbc0 con 0x7f1c24070ef0 2023-03-03T06:26:11.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.303+0000 7f1c2b445700 1 -- 172.21.15.3:0/49716064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c240cbf60 con 0x7f1c24070ef0 2023-03-03T06:26:11.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.303+0000 7f1c2b445700 1 -- 172.21.15.3:0/49716064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c240cd130 con 0x7f1c24070ef0 2023-03-03T06:26:11.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.303+0000 7f1c2b445700 1 -- 172.21.15.3:0/49716064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2415c4f0 con 0x7f1c24070ef0 2023-03-03T06:26:11.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.307+0000 7f1c1affd700 1 -- 172.21.15.3:0/49716064 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1c14019030 con 0x7f1c24070ef0 2023-03-03T06:26:11.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.307+0000 7f1c1affd700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1c08064f10 0x7f1c080673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:11.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.307+0000 7f1c1affd700 1 -- 172.21.15.3:0/49716064 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1c140822e0 con 0x7f1c24070ef0 2023-03-03T06:26:11.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.307+0000 7f1c2a443700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1c08064f10 0x7f1c080673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:11.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.311+0000 7f1c2a443700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1c08064f10 0x7f1c080673c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1c200090b0 tx=0x7f1c20013040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:11.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.315+0000 7f1c1affd700 1 -- 172.21.15.3:0/49716064 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1c14050060 con 0x7f1c24070ef0 2023-03-03T06:26:11.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.495+0000 7f1c2b445700 1 -- 172.21.15.3:0/49716064 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c241f5c40 con 0x7f1c24070ef0 2023-03-03T06:26:11.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.495+0000 7f1c1affd700 1 -- 172.21.15.3:0/49716064 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c241f5c40 con 0x7f1c24070ef0 2023-03-03T06:26:11.501 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:11.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.499+0000 7f1c2b445700 1 -- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1c08064f10 msgr2=0x7f1c080673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.499+0000 7f1c2b445700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1c08064f10 0x7f1c080673c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1c200090b0 tx=0x7f1c20013040 comp rx=0 tx=0).stop 2023-03-03T06:26:11.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.499+0000 7f1c2b445700 1 -- 172.21.15.3:0/49716064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c24070ef0 msgr2=0x7f1c24153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.499+0000 7f1c2b445700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c24070ef0 0x7f1c24153520 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400e9d0 tx=0x7f1c1400ed90 comp rx=0 tx=0).stop 2023-03-03T06:26:11.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.499+0000 7f1c2b445700 1 -- 172.21.15.3:0/49716064 shutdown_connections 2023-03-03T06:26:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.499+0000 7f1c2b445700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1c08064f10 0x7f1c080673c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.499+0000 7f1c2b445700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1c240cb9a0 0x7f1c24153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.499+0000 7f1c2b445700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c24070ef0 0x7f1c24153520 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.499+0000 7f1c2b445700 1 --2- 172.21.15.3:0/49716064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c2406a780 0x7f1c2414e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.499+0000 7f1c2b445700 1 -- 172.21.15.3:0/49716064 >> 172.21.15.3:0/49716064 conn(0x7f1c240607c0 msgr2=0x7f1c24060b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.499+0000 7f1c2b445700 1 -- 172.21.15.3:0/49716064 shutdown_connections 2023-03-03T06:26:11.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.499+0000 7f1c2b445700 1 -- 172.21.15.3:0/49716064 wait complete. 2023-03-03T06:26:11.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-03T06:26:11.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.735+0000 7f3862551700 1 Processor -- start 2023-03-03T06:26:11.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.739+0000 7f3862551700 1 -- start start 2023-03-03T06:26:11.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.739+0000 7f3862551700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f385c0b4c70 0x7f385c0d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:11.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.739+0000 7f3862551700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385c0d1260 con 0x7f385c15c3c0 2023-03-03T06:26:11.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.739+0000 7f3862551700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385c0cd890 con 0x7f385c0cfd40 2023-03-03T06:26:11.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.739+0000 7f3862551700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385c15c790 con 0x7f385c0b4c70 2023-03-03T06:26:11.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.739+0000 7f386154f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f385c15c3c0 0x7f385c0b5910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52394/0 (socket says 172.21.15.3:52394) 2023-03-03T06:26:11.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.739+0000 7f386154f700 1 -- 172.21.15.3:0/426805572 learned_addr learned my addr 172.21.15.3:0/426805572 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:11.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.739+0000 7f3861d50700 1 --2- 172.21.15.3:0/426805572 >> v2:172.21.15.32:3300/0 conn(0x7f385c0b4c70 0x7f385c0d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:11.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.743+0000 7f3853fff700 1 -- 172.21.15.3:0/426805572 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449869893 0 0) 0x7f385c0d1260 con 0x7f385c15c3c0 2023-03-03T06:26:11.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.743+0000 7f3853fff700 1 -- 172.21.15.3:0/426805572 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38480035d0 con 0x7f385c15c3c0 2023-03-03T06:26:11.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.743+0000 7f3853fff700 1 -- 172.21.15.3:0/426805572 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952970543 0 0) 0x7f385c0cd890 con 0x7f385c0cfd40 2023-03-03T06:26:11.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.743+0000 7f3853fff700 1 -- 172.21.15.3:0/426805572 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f385c0d1260 con 0x7f385c0cfd40 2023-03-03T06:26:11.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.743+0000 7f3861d50700 1 -- 172.21.15.3:0/426805572 >> v1:172.21.15.32:6789/0 conn(0x7f385c0cfd40 legacy=0x7f385c0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.743+0000 7f3861d50700 1 -- 172.21.15.3:0/426805572 >> v1:172.21.15.3:6789/0 conn(0x7f385c15c3c0 legacy=0x7f385c0b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.743+0000 7f3861d50700 1 -- 172.21.15.3:0/426805572 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c0cd2d0 con 0x7f385c0b4c70 2023-03-03T06:26:11.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.743+0000 7f3861d50700 1 --2- 172.21.15.3:0/426805572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c0b4c70 0x7f385c0d1a20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3854009d90 tx=0x7f385400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40d0cc1314c9ff93 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:11.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.743+0000 7f3853fff700 1 -- 172.21.15.3:0/426805572 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400e070 con 0x7f385c0b4c70 2023-03-03T06:26:11.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.743+0000 7f3853fff700 1 -- 172.21.15.3:0/426805572 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385400bb20 con 0x7f385c0b4c70 2023-03-03T06:26:11.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.743+0000 7f3853fff700 1 -- 172.21.15.3:0/426805572 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854012430 con 0x7f385c0b4c70 2023-03-03T06:26:11.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.747+0000 7f3862551700 1 -- 172.21.15.3:0/426805572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c0b4c70 msgr2=0x7f385c0d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.747+0000 7f3862551700 1 --2- 172.21.15.3:0/426805572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c0b4c70 0x7f385c0d1a20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3854009d90 tx=0x7f385400b330 comp rx=0 tx=0).stop 2023-03-03T06:26:11.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.747+0000 7f3862551700 1 -- 172.21.15.3:0/426805572 shutdown_connections 2023-03-03T06:26:11.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.747+0000 7f3862551700 1 --2- 172.21.15.3:0/426805572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c0b4c70 0x7f385c0d1a20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.747+0000 7f3862551700 1 -- 172.21.15.3:0/426805572 >> 172.21.15.3:0/426805572 conn(0x7f385c070700 msgr2=0x7f385c070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:11.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.747+0000 7f3862551700 1 -- 172.21.15.3:0/426805572 shutdown_connections 2023-03-03T06:26:11.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.747+0000 7f3862551700 1 -- 172.21.15.3:0/426805572 wait complete. 2023-03-03T06:26:11.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.751+0000 7f3862551700 1 Processor -- start 2023-03-03T06:26:11.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.751+0000 7f3862551700 1 -- start start 2023-03-03T06:26:11.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.751+0000 7f3862551700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385c0b4c70 0x7f385c1507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.751+0000 7f3862551700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c0cfd40 0x7f385c15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.751+0000 7f3862551700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f385c15c3c0 0x7f385c15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.751+0000 7f3862551700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c150d00 con 0x7f385c0b4c70 2023-03-03T06:26:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.751+0000 7f3862551700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0ca660 con 0x7f385c0cfd40 2023-03-03T06:26:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.751+0000 7f3862551700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0ca940 con 0x7f385c15c3c0 2023-03-03T06:26:11.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.751+0000 7f386154f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385c0b4c70 0x7f385c1507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:11.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.751+0000 7f386154f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385c0b4c70 0x7f385c1507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44856/0 (socket says 172.21.15.3:44856) 2023-03-03T06:26:11.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.751+0000 7f386154f700 1 -- 172.21.15.3:0/102691308 learned_addr learned my addr 172.21.15.3:0/102691308 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f3861d50700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f385c15c3c0 0x7f385c15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f3860d4e700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c0cfd40 0x7f385c15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f386154f700 1 -- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f385c15c3c0 msgr2=0x7f385c15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f386154f700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f385c15c3c0 0x7f385c15cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f386154f700 1 -- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c0cfd40 msgr2=0x7f385c15a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f386154f700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c0cfd40 0x7f385c15a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f386154f700 1 -- 172.21.15.3:0/102691308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3854009a70 con 0x7f385c0b4c70 2023-03-03T06:26:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f3860d4e700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c0cfd40 0x7f385c15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f386154f700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385c0b4c70 0x7f385c1507c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f385800c4d0 tx=0x7f385800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:11.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f3851ffb700 1 -- 172.21.15.3:0/102691308 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858005d90 con 0x7f385c0b4c70 2023-03-03T06:26:11.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f3851ffb700 1 -- 172.21.15.3:0/102691308 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38580070d0 con 0x7f385c0b4c70 2023-03-03T06:26:11.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f3862551700 1 -- 172.21.15.3:0/102691308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385c0d1500 con 0x7f385c0b4c70 2023-03-03T06:26:11.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f3851ffb700 1 -- 172.21.15.3:0/102691308 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858007460 con 0x7f385c0b4c70 2023-03-03T06:26:11.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.755+0000 7f3862551700 1 -- 172.21.15.3:0/102691308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385c0cd5b0 con 0x7f385c0b4c70 2023-03-03T06:26:11.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.759+0000 7f3862551700 1 -- 172.21.15.3:0/102691308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385c0b6020 con 0x7f385c0b4c70 2023-03-03T06:26:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.759+0000 7f3851ffb700 1 -- 172.21.15.3:0/102691308 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3858019030 con 0x7f385c0b4c70 2023-03-03T06:26:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.763+0000 7f3851ffb700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3840064ee0 0x7f3840067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.763+0000 7f3851ffb700 1 -- 172.21.15.3:0/102691308 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3858081610 con 0x7f385c0b4c70 2023-03-03T06:26:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.763+0000 7f3860d4e700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3840064ee0 0x7f3840067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:11.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.763+0000 7f3860d4e700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3840064ee0 0x7f3840067390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f385c159ff0 tx=0x7f384c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:11.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.767+0000 7f3851ffb700 1 -- 172.21.15.3:0/102691308 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f385804f390 con 0x7f385c0b4c70 2023-03-03T06:26:11.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.943+0000 7f3862551700 1 -- 172.21.15.3:0/102691308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f385c0b8820 con 0x7f385c0b4c70 2023-03-03T06:26:11.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.943+0000 7f3851ffb700 1 -- 172.21.15.3:0/102691308 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f38580529b0 con 0x7f385c0b4c70 2023-03-03T06:26:11.949 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-03-03T06:26:11.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.947+0000 7f3862551700 1 -- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3840064ee0 msgr2=0x7f3840067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.947+0000 7f3862551700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3840064ee0 0x7f3840067390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f385c159ff0 tx=0x7f384c009040 comp rx=0 tx=0).stop 2023-03-03T06:26:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.947+0000 7f3862551700 1 -- 172.21.15.3:0/102691308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385c0b4c70 msgr2=0x7f385c1507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.947+0000 7f3862551700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385c0b4c70 0x7f385c1507c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f385800c4d0 tx=0x7f385800c7e0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.947+0000 7f3862551700 1 -- 172.21.15.3:0/102691308 shutdown_connections 2023-03-03T06:26:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.947+0000 7f3862551700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3840064ee0 0x7f3840067390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.947+0000 7f3862551700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f385c15c3c0 0x7f385c15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.947+0000 7f3862551700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f385c0cfd40 0x7f385c15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.947+0000 7f3862551700 1 --2- 172.21.15.3:0/102691308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f385c0b4c70 0x7f385c1507c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.947+0000 7f3862551700 1 -- 172.21.15.3:0/102691308 >> 172.21.15.3:0/102691308 conn(0x7f385c070700 msgr2=0x7f385c0b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.951+0000 7f3862551700 1 -- 172.21.15.3:0/102691308 shutdown_connections 2023-03-03T06:26:11.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:11.951+0000 7f3862551700 1 -- 172.21.15.3:0/102691308 wait complete. 2023-03-03T06:26:11.986 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-03T06:26:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.191+0000 7f31848f9700 1 Processor -- start 2023-03-03T06:26:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.191+0000 7f31848f9700 1 -- start start 2023-03-03T06:26:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f31848f9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3180159240 0x7f31800cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:12.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f31848f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31800cdc50 0x7f318015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:12.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f31848f9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3180070020 con 0x7f31800ce150 2023-03-03T06:26:12.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f31848f9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3180070300 con 0x7f3180159240 2023-03-03T06:26:12.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f31848f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318015ce30 con 0x7f31800cdc50 2023-03-03T06:26:12.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f317ed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3180159240 0x7f31800cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:12.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f317ed9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3180159240 0x7f31800cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44872/0 (socket says 172.21.15.3:44872) 2023-03-03T06:26:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f317ed9d700 1 -- 172.21.15.3:0/24746498 learned_addr learned my addr 172.21.15.3:0/24746498 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f317e59c700 1 --2- 172.21.15.3:0/24746498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31800cdc50 0x7f318015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f317ed9d700 1 -- 172.21.15.3:0/24746498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31800cdc50 msgr2=0x7f318015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f317ed9d700 1 --2- 172.21.15.3:0/24746498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31800cdc50 0x7f318015d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f317ed9d700 1 -- 172.21.15.3:0/24746498 >> v1:172.21.15.32:6789/0 conn(0x7f31800ce150 legacy=0x7f31800ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f317ed9d700 1 -- 172.21.15.3:0/24746498 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318015d110 con 0x7f3180159240 2023-03-03T06:26:12.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f317e59c700 1 --2- 172.21.15.3:0/24746498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31800cdc50 0x7f318015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.195+0000 7f317ed9d700 1 --2- 172.21.15.3:0/24746498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3180159240 0x7f31800cd390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3170009900 tx=0x7f3170009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e975d298c91933 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.199+0000 7f317dd9b700 1 -- 172.21.15.3:0/24746498 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3170011830 con 0x7f3180159240 2023-03-03T06:26:12.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.199+0000 7f317dd9b700 1 -- 172.21.15.3:0/24746498 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3170011990 con 0x7f3180159240 2023-03-03T06:26:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.199+0000 7f317dd9b700 1 -- 172.21.15.3:0/24746498 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3170011d40 con 0x7f3180159240 2023-03-03T06:26:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.199+0000 7f31848f9700 1 -- 172.21.15.3:0/24746498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3180159240 msgr2=0x7f31800cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:12.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.199+0000 7f31848f9700 1 --2- 172.21.15.3:0/24746498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3180159240 0x7f31800cd390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3170009900 tx=0x7f3170009c10 comp rx=0 tx=0).stop 2023-03-03T06:26:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.199+0000 7f31848f9700 1 -- 172.21.15.3:0/24746498 shutdown_connections 2023-03-03T06:26:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.199+0000 7f31848f9700 1 --2- 172.21.15.3:0/24746498 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31800cdc50 0x7f318015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.199+0000 7f31848f9700 1 --2- 172.21.15.3:0/24746498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3180159240 0x7f31800cd390 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.199+0000 7f31848f9700 1 -- 172.21.15.3:0/24746498 >> 172.21.15.3:0/24746498 conn(0x7f3180069d70 msgr2=0x7f318006a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.199+0000 7f31848f9700 1 -- 172.21.15.3:0/24746498 shutdown_connections 2023-03-03T06:26:12.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.199+0000 7f31848f9700 1 -- 172.21.15.3:0/24746498 wait complete. 2023-03-03T06:26:12.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.203+0000 7f31848f9700 1 Processor -- start 2023-03-03T06:26:12.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.203+0000 7f31848f9700 1 -- start start 2023-03-03T06:26:12.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f31848f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31800cdc50 0x7f31801f1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:12.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f31848f9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31800ce150 0x7f31800b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:12.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f31848f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3180159240 0x7f31800b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:12.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f31848f9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31801597a0 con 0x7f31800ce150 2023-03-03T06:26:12.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f31848f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3180159d60 con 0x7f3180159240 2023-03-03T06:26:12.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f31848f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31801e48d0 con 0x7f31800cdc50 2023-03-03T06:26:12.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f317e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31800ce150 0x7f31800b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:12.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f317e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31800ce150 0x7f31800b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44886/0 (socket says 172.21.15.3:44886) 2023-03-03T06:26:12.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f317e59c700 1 -- 172.21.15.3:0/82539660 learned_addr learned my addr 172.21.15.3:0/82539660 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:12.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f317ed9d700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31800cdc50 0x7f31801f1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f317f59e700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3180159240 0x7f31800b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f317e59c700 1 -- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31800cdc50 msgr2=0x7f31801f1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f317e59c700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31800cdc50 0x7f31801f1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f317e59c700 1 -- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3180159240 msgr2=0x7f31800b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f317e59c700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3180159240 0x7f31800b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f317e59c700 1 -- 172.21.15.3:0/82539660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31740094b0 con 0x7f31800ce150 2023-03-03T06:26:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f317e59c700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31800ce150 0x7f31800b4440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f317400ec10 tx=0x7f317400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f31677fe700 1 -- 172.21.15.3:0/82539660 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317400c850 con 0x7f31800ce150 2023-03-03T06:26:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f31677fe700 1 -- 172.21.15.3:0/82539660 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317400c9b0 con 0x7f31800ce150 2023-03-03T06:26:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f31848f9700 1 -- 172.21.15.3:0/82539660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3170009510 con 0x7f31800ce150 2023-03-03T06:26:12.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f31677fe700 1 -- 172.21.15.3:0/82539660 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317400cd40 con 0x7f31800ce150 2023-03-03T06:26:12.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.207+0000 7f31848f9700 1 -- 172.21.15.3:0/82539660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31800705e0 con 0x7f31800ce150 2023-03-03T06:26:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.211+0000 7f31848f9700 1 -- 172.21.15.3:0/82539660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3180159610 con 0x7f31800ce150 2023-03-03T06:26:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.211+0000 7f31677fe700 1 -- 172.21.15.3:0/82539660 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3174023070 con 0x7f31800ce150 2023-03-03T06:26:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.215+0000 7f31677fe700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f316c064ee0 0x7f316c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:12.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.215+0000 7f31677fe700 1 -- 172.21.15.3:0/82539660 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f31740824f0 con 0x7f31800ce150 2023-03-03T06:26:12.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.215+0000 7f317ed9d700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f316c064ee0 0x7f316c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:12.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.215+0000 7f317ed9d700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f316c064ee0 0x7f316c067390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3170014040 tx=0x7f3170008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:12.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.219+0000 7f31677fe700 1 -- 172.21.15.3:0/82539660 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f3174050270 con 0x7f31800ce150 2023-03-03T06:26:12.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.395+0000 7f31848f9700 1 -- 172.21.15.3:0/82539660 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f31800cd8d0 con 0x7f31800ce150 2023-03-03T06:26:12.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.399+0000 7f31677fe700 1 -- 172.21.15.3:0/82539660 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f317400cea0 con 0x7f31800ce150 2023-03-03T06:26:12.402 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.399+0000 7f31848f9700 1 -- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f316c064ee0 msgr2=0x7f316c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.399+0000 7f31848f9700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f316c064ee0 0x7f316c067390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3170014040 tx=0x7f3170008030 comp rx=0 tx=0).stop 2023-03-03T06:26:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.399+0000 7f31848f9700 1 -- 172.21.15.3:0/82539660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31800ce150 msgr2=0x7f31800b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.399+0000 7f31848f9700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31800ce150 0x7f31800b4440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f317400ec10 tx=0x7f317400ef20 comp rx=0 tx=0).stop 2023-03-03T06:26:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.399+0000 7f31848f9700 1 -- 172.21.15.3:0/82539660 shutdown_connections 2023-03-03T06:26:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.399+0000 7f31848f9700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f316c064ee0 0x7f316c067390 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.399+0000 7f31848f9700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3180159240 0x7f31800b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.399+0000 7f31848f9700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31800ce150 0x7f31800b4440 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.399+0000 7f31848f9700 1 --2- 172.21.15.3:0/82539660 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31800cdc50 0x7f31801f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.399+0000 7f31848f9700 1 -- 172.21.15.3:0/82539660 >> 172.21.15.3:0/82539660 conn(0x7f3180069d70 msgr2=0x7f31800ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.399+0000 7f31848f9700 1 -- 172.21.15.3:0/82539660 shutdown_connections 2023-03-03T06:26:12.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.403+0000 7f31848f9700 1 -- 172.21.15.3:0/82539660 wait complete. 2023-03-03T06:26:12.443 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:12.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.655+0000 7fd86e9d2700 1 Processor -- start 2023-03-03T06:26:12.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.655+0000 7fd86e9d2700 1 -- start start 2023-03-03T06:26:12.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86e9d2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8680adf20 0x7fd8680b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:12.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86e9d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8680cb890 0x7fd86815e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:12.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86e9d2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8680c9a20 con 0x7fd86815dec0 2023-03-03T06:26:12.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86e9d2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8680ae2f0 con 0x7fd8680adf20 2023-03-03T06:26:12.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86e9d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680c9fe0 con 0x7fd8680cb890 2023-03-03T06:26:12.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86d1cf700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd86815dec0 0x7fd8680cb180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52400/0 (socket says 172.21.15.3:52400) 2023-03-03T06:26:12.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86d1cf700 1 -- 172.21.15.3:0/1389831508 learned_addr learned my addr 172.21.15.3:0/1389831508 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:12.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86d9d0700 1 --2- 172.21.15.3:0/1389831508 >> v2:172.21.15.32:3300/0 conn(0x7fd8680adf20 0x7fd8680b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:12.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86e1d1700 1 --2- 172.21.15.3:0/1389831508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8680cb890 0x7fd86815e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:12.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86c9ce700 1 -- 172.21.15.3:0/1389831508 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946086227 0 0) 0x7fd8680c9a20 con 0x7fd86815dec0 2023-03-03T06:26:12.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86d9d0700 1 -- 172.21.15.3:0/1389831508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8680cb890 msgr2=0x7fd86815e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:12.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86d9d0700 1 --2- 172.21.15.3:0/1389831508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8680cb890 0x7fd86815e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86d9d0700 1 -- 172.21.15.3:0/1389831508 >> v1:172.21.15.3:6789/0 conn(0x7fd86815dec0 legacy=0x7fd8680cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:12.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.659+0000 7fd86d9d0700 1 -- 172.21.15.3:0/1389831508 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86815a7d0 con 0x7fd8680adf20 2023-03-03T06:26:12.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.663+0000 7fd86d9d0700 1 --2- 172.21.15.3:0/1389831508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8680adf20 0x7fd8680b1630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd864009850 tx=0x7fd864009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ddcba2e1513fd3a server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:12.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.663+0000 7fd86c9ce700 1 -- 172.21.15.3:0/1389831508 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd864011830 con 0x7fd8680adf20 2023-03-03T06:26:12.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.663+0000 7fd86c9ce700 1 -- 172.21.15.3:0/1389831508 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd864011990 con 0x7fd8680adf20 2023-03-03T06:26:12.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.663+0000 7fd86c9ce700 1 -- 172.21.15.3:0/1389831508 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd864011d40 con 0x7fd8680adf20 2023-03-03T06:26:12.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.663+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1389831508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8680adf20 msgr2=0x7fd8680b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:12.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.663+0000 7fd86e9d2700 1 --2- 172.21.15.3:0/1389831508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8680adf20 0x7fd8680b1630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd864009850 tx=0x7fd864009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:12.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.663+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1389831508 shutdown_connections 2023-03-03T06:26:12.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.663+0000 7fd86e9d2700 1 --2- 172.21.15.3:0/1389831508 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd8680cb890 0x7fd86815e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.663+0000 7fd86e9d2700 1 --2- 172.21.15.3:0/1389831508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8680adf20 0x7fd8680b1630 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.663+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1389831508 >> 172.21.15.3:0/1389831508 conn(0x7fd868070a30 msgr2=0x7fd868070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:12.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.663+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1389831508 shutdown_connections 2023-03-03T06:26:12.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.663+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1389831508 wait complete. 2023-03-03T06:26:12.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.667+0000 7fd86e9d2700 1 Processor -- start 2023-03-03T06:26:12.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.667+0000 7fd86e9d2700 1 -- start start 2023-03-03T06:26:12.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.667+0000 7fd86e9d2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8680adf20 0x7fd86814d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:12.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.667+0000 7fd86e9d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8680cb890 0x7fd8680b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:12.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.667+0000 7fd86e9d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86815dec0 0x7fd8681f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:12.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.667+0000 7fd86e9d2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680b1e80 con 0x7fd8680adf20 2023-03-03T06:26:12.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.667+0000 7fd86e9d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680b2160 con 0x7fd8680cb890 2023-03-03T06:26:12.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.667+0000 7fd86e9d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680b2440 con 0x7fd86815dec0 2023-03-03T06:26:12.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.667+0000 7fd86d9d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8680adf20 0x7fd86814d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:12.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd86d9d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8680adf20 0x7fd86814d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44902/0 (socket says 172.21.15.3:44902) 2023-03-03T06:26:12.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd86d9d0700 1 -- 172.21.15.3:0/1131311231 learned_addr learned my addr 172.21.15.3:0/1131311231 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:12.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd86e1d1700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86815dec0 0x7fd8681f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:12.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd86d1cf700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8680cb890 0x7fd8680b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:12.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd86e1d1700 1 -- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8680cb890 msgr2=0x7fd8680b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:12.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd86e1d1700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8680cb890 0x7fd8680b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd86e1d1700 1 -- 172.21.15.3:0/1131311231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8680adf20 msgr2=0x7fd86814d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:12.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd86e1d1700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8680adf20 0x7fd86814d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd86e1d1700 1 -- 172.21.15.3:0/1131311231 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8640094b0 con 0x7fd86815dec0 2023-03-03T06:26:12.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd86d9d0700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8680adf20 0x7fd86814d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:12.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd86e1d1700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86815dec0 0x7fd8681f11c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd86000ef90 tx=0x7fd86000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:12.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd85e7fc700 1 -- 172.21.15.3:0/1131311231 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86000cc90 con 0x7fd86815dec0 2023-03-03T06:26:12.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd85e7fc700 1 -- 172.21.15.3:0/1131311231 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86000f040 con 0x7fd86815dec0 2023-03-03T06:26:12.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd85e7fc700 1 -- 172.21.15.3:0/1131311231 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8600146c0 con 0x7fd86815dec0 2023-03-03T06:26:12.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1131311231 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86815ebf0 con 0x7fd86815dec0 2023-03-03T06:26:12.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.671+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1131311231 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86815e910 con 0x7fd86815dec0 2023-03-03T06:26:12.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.675+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1131311231 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8680c9c70 con 0x7fd86815dec0 2023-03-03T06:26:12.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.675+0000 7fd85e7fc700 1 -- 172.21.15.3:0/1131311231 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd860014820 con 0x7fd86815dec0 2023-03-03T06:26:12.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.679+0000 7fd85e7fc700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd84c064ee0 0x7fd84c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:12.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.679+0000 7fd85e7fc700 1 -- 172.21.15.3:0/1131311231 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd860082ed0 con 0x7fd86815dec0 2023-03-03T06:26:12.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.679+0000 7fd86d9d0700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd84c064ee0 0x7fd84c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:12.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.679+0000 7fd86d9d0700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd84c064ee0 0x7fd84c067390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd864009ba0 tx=0x7fd864008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:12.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.687+0000 7fd85e7fc700 1 -- 172.21.15.3:0/1131311231 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd860050bc0 con 0x7fd86815dec0 2023-03-03T06:26:12.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.867+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1131311231 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8680c9400 con 0x7fd86815dec0 2023-03-03T06:26:12.874 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.871+0000 7fd85e7fc700 1 -- 172.21.15.3:0/1131311231 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19652 (secure 0 0 0) 0x7fd860018070 con 0x7fd86815dec0 2023-03-03T06:26:12.874 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:12.875 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:53.877945+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.875+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd84c064ee0 msgr2=0x7fd84c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:12.879 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.875+0000 7fd86e9d2700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd84c064ee0 0x7fd84c067390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd864009ba0 tx=0x7fd864008030 comp rx=0 tx=0).stop 2023-03-03T06:26:12.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.875+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86815dec0 msgr2=0x7fd8681f11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:12.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.875+0000 7fd86e9d2700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86815dec0 0x7fd8681f11c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd86000ef90 tx=0x7fd86000c3a0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.875+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1131311231 shutdown_connections 2023-03-03T06:26:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.875+0000 7fd86e9d2700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd84c064ee0 0x7fd84c067390 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.875+0000 7fd86e9d2700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd86815dec0 0x7fd8681f11c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.875+0000 7fd86e9d2700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8680cb890 0x7fd8680b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.875+0000 7fd86e9d2700 1 --2- 172.21.15.3:0/1131311231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8680adf20 0x7fd86814d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.875+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1131311231 >> 172.21.15.3:0/1131311231 conn(0x7fd868070a30 msgr2=0x7fd8680accc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:12.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.875+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1131311231 shutdown_connections 2023-03-03T06:26:12.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:12.875+0000 7fd86e9d2700 1 -- 172.21.15.3:0/1131311231 wait complete. 2023-03-03T06:26:12.911 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-03T06:25:11.468534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-03T06:25:12.813283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-03T06:25:16.382800+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-03T06:26:12.912 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:26:13.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.119+0000 7f5c6a05a700 1 Processor -- start 2023-03-03T06:26:13.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.119+0000 7f5c6a05a700 1 -- start start 2023-03-03T06:26:13.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.123+0000 7f5c6a05a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c64158400 0x7f5c641587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:13.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.123+0000 7f5c6a05a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6415c230 con 0x7f5c640cb9a0 2023-03-03T06:26:13.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.123+0000 7f5c6a05a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c640cd130 con 0x7f5c640710c0 2023-03-03T06:26:13.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.123+0000 7f5c6a05a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c640caac0 con 0x7f5c64158400 2023-03-03T06:26:13.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.123+0000 7f5c68857700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5c640cb9a0 0x7f5c640709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52402/0 (socket says 172.21.15.3:52402) 2023-03-03T06:26:13.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.123+0000 7f5c68857700 1 -- 172.21.15.3:0/3384273953 learned_addr learned my addr 172.21.15.3:0/3384273953 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:13.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.123+0000 7f5c69058700 1 --2- 172.21.15.3:0/3384273953 >> v2:172.21.15.3:3300/0 conn(0x7f5c64158400 0x7f5c641587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:13.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.123+0000 7f5c5bfff700 1 -- 172.21.15.3:0/3384273953 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053747606 0 0) 0x7f5c6415c230 con 0x7f5c640cb9a0 2023-03-03T06:26:13.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.123+0000 7f5c5bfff700 1 -- 172.21.15.3:0/3384273953 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c500035d0 con 0x7f5c640cb9a0 2023-03-03T06:26:13.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.123+0000 7f5c69058700 1 -- 172.21.15.3:0/3384273953 >> v1:172.21.15.32:6789/0 conn(0x7f5c640710c0 legacy=0x7f5c640ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:13.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.123+0000 7f5c69058700 1 -- 172.21.15.3:0/3384273953 >> v1:172.21.15.3:6789/0 conn(0x7f5c640cb9a0 legacy=0x7f5c640709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:13.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.123+0000 7f5c69058700 1 -- 172.21.15.3:0/3384273953 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c640cbf60 con 0x7f5c64158400 2023-03-03T06:26:13.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.127+0000 7f5c69058700 1 --2- 172.21.15.3:0/3384273953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c64158400 0x7f5c641587d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000a9f0 tx=0x7f5c6000ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e3dd75c434e162 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:13.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.127+0000 7f5c5bfff700 1 -- 172.21.15.3:0/3384273953 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000cb70 con 0x7f5c64158400 2023-03-03T06:26:13.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.127+0000 7f5c5bfff700 1 -- 172.21.15.3:0/3384273953 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6000ae90 con 0x7f5c64158400 2023-03-03T06:26:13.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.127+0000 7f5c5bfff700 1 -- 172.21.15.3:0/3384273953 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60018400 con 0x7f5c64158400 2023-03-03T06:26:13.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.127+0000 7f5c6a05a700 1 -- 172.21.15.3:0/3384273953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c64158400 msgr2=0x7f5c641587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:13.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.127+0000 7f5c6a05a700 1 --2- 172.21.15.3:0/3384273953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c64158400 0x7f5c641587d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000a9f0 tx=0x7f5c6000ad00 comp rx=0 tx=0).stop 2023-03-03T06:26:13.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.127+0000 7f5c6a05a700 1 -- 172.21.15.3:0/3384273953 shutdown_connections 2023-03-03T06:26:13.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.127+0000 7f5c6a05a700 1 --2- 172.21.15.3:0/3384273953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c64158400 0x7f5c641587d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.127+0000 7f5c6a05a700 1 -- 172.21.15.3:0/3384273953 >> 172.21.15.3:0/3384273953 conn(0x7f5c640607c0 msgr2=0x7f5c64060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:13.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.127+0000 7f5c6a05a700 1 -- 172.21.15.3:0/3384273953 shutdown_connections 2023-03-03T06:26:13.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.127+0000 7f5c6a05a700 1 -- 172.21.15.3:0/3384273953 wait complete. 2023-03-03T06:26:13.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.131+0000 7f5c6a05a700 1 Processor -- start 2023-03-03T06:26:13.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.131+0000 7f5c6a05a700 1 -- start start 2023-03-03T06:26:13.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.131+0000 7f5c6a05a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c640710c0 0x7f5c640ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:13.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.131+0000 7f5c6a05a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c640cb9a0 0x7f5c640aba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:13.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.131+0000 7f5c6a05a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64158400 0x7f5c640abf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:13.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.131+0000 7f5c6a05a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640ad350 con 0x7f5c640cb9a0 2023-03-03T06:26:13.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.131+0000 7f5c6a05a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640ac490 con 0x7f5c64158400 2023-03-03T06:26:13.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.131+0000 7f5c6a05a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640ac740 con 0x7f5c640710c0 2023-03-03T06:26:13.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.131+0000 7f5c68857700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c640cb9a0 0x7f5c640aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:13.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c68857700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c640cb9a0 0x7f5c640aba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44910/0 (socket says 172.21.15.3:44910) 2023-03-03T06:26:13.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c69058700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c640710c0 0x7f5c640ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:13.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c68857700 1 -- 172.21.15.3:0/684497323 learned_addr learned my addr 172.21.15.3:0/684497323 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:13.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c69859700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64158400 0x7f5c640abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:13.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c68857700 1 -- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c640710c0 msgr2=0x7f5c640ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:13.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c68857700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c640710c0 0x7f5c640ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c68857700 1 -- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64158400 msgr2=0x7f5c640abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:13.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c68857700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64158400 0x7f5c640abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c68857700 1 -- 172.21.15.3:0/684497323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6000a650 con 0x7f5c640cb9a0 2023-03-03T06:26:13.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c69859700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64158400 0x7f5c640abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:13.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c68857700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c640cb9a0 0x7f5c640aba10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400c700 tx=0x7f5c5400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:13.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c59ffb700 1 -- 172.21.15.3:0/684497323 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54005d90 con 0x7f5c640cb9a0 2023-03-03T06:26:13.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c59ffb700 1 -- 172.21.15.3:0/684497323 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c540070d0 con 0x7f5c640cb9a0 2023-03-03T06:26:13.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c59ffb700 1 -- 172.21.15.3:0/684497323 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54007480 con 0x7f5c640cb9a0 2023-03-03T06:26:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c6a05a700 1 -- 172.21.15.3:0/684497323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6415c7d0 con 0x7f5c640cb9a0 2023-03-03T06:26:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.135+0000 7f5c6a05a700 1 -- 172.21.15.3:0/684497323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6415c4f0 con 0x7f5c640cb9a0 2023-03-03T06:26:13.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.139+0000 7f5c6a05a700 1 -- 172.21.15.3:0/684497323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6415cab0 con 0x7f5c640cb9a0 2023-03-03T06:26:13.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.139+0000 7f5c59ffb700 1 -- 172.21.15.3:0/684497323 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5c54019030 con 0x7f5c640cb9a0 2023-03-03T06:26:13.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.143+0000 7f5c59ffb700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5c48064ee0 0x7f5c48067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:13.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.143+0000 7f5c59ffb700 1 -- 172.21.15.3:0/684497323 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5c540816a0 con 0x7f5c640cb9a0 2023-03-03T06:26:13.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.143+0000 7f5c69058700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5c48064ee0 0x7f5c48067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:13.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.147+0000 7f5c69058700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5c48064ee0 0x7f5c48067390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000a1c0 tx=0x7f5c6001b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:13.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.147+0000 7f5c59ffb700 1 -- 172.21.15.3:0/684497323 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5c5404f420 con 0x7f5c640cb9a0 2023-03-03T06:26:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.323+0000 7f5c6a05a700 1 -- 172.21.15.3:0/684497323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c64073e90 con 0x7f5c640cb9a0 2023-03-03T06:26:13.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.323+0000 7f5c59ffb700 1 -- 172.21.15.3:0/684497323 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c54052a40 con 0x7f5c640cb9a0 2023-03-03T06:26:13.330 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:26:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.327+0000 7f5c6a05a700 1 -- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5c48064ee0 msgr2=0x7f5c48067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.327+0000 7f5c6a05a700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5c48064ee0 0x7f5c48067390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000a1c0 tx=0x7f5c6001b040 comp rx=0 tx=0).stop 2023-03-03T06:26:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.327+0000 7f5c6a05a700 1 -- 172.21.15.3:0/684497323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c640cb9a0 msgr2=0x7f5c640aba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:13.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.327+0000 7f5c6a05a700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c640cb9a0 0x7f5c640aba10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400c700 tx=0x7f5c5400ca10 comp rx=0 tx=0).stop 2023-03-03T06:26:13.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.327+0000 7f5c6a05a700 1 -- 172.21.15.3:0/684497323 shutdown_connections 2023-03-03T06:26:13.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.327+0000 7f5c6a05a700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5c48064ee0 0x7f5c48067390 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.327+0000 7f5c6a05a700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c64158400 0x7f5c640abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.327+0000 7f5c6a05a700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c640cb9a0 0x7f5c640aba10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.327+0000 7f5c6a05a700 1 --2- 172.21.15.3:0/684497323 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5c640710c0 0x7f5c640ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.327+0000 7f5c6a05a700 1 -- 172.21.15.3:0/684497323 >> 172.21.15.3:0/684497323 conn(0x7f5c640607c0 msgr2=0x7f5c6406c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.327+0000 7f5c6a05a700 1 -- 172.21.15.3:0/684497323 shutdown_connections 2023-03-03T06:26:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.327+0000 7f5c6a05a700 1 -- 172.21.15.3:0/684497323 wait complete. 2023-03-03T06:26:13.363 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:26:13.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.571+0000 7f1edf3c3700 1 Processor -- start 2023-03-03T06:26:13.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.571+0000 7f1edf3c3700 1 -- start start 2023-03-03T06:26:13.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.575+0000 7f1edf3c3700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1ed815d6e0 0x7f1ed80cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:13.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.575+0000 7f1edf3c3700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ed80cfd80 0x7f1ed80d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:13.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.575+0000 7f1edf3c3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed80cf3b0 con 0x7f1ed8156ba0 2023-03-03T06:26:13.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.575+0000 7f1edf3c3700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed8156f70 con 0x7f1ed815d6e0 2023-03-03T06:26:13.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.575+0000 7f1edf3c3700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed80cd320 con 0x7f1ed80cfd80 2023-03-03T06:26:13.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.575+0000 7f1eddbc0700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1ed815d6e0 0x7f1ed80cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:13.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.575+0000 7f1ede3c1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1ed8156ba0 0x7f1ed8071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52416/0 (socket says 172.21.15.3:52416) 2023-03-03T06:26:13.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.575+0000 7f1ede3c1700 1 -- 172.21.15.3:0/2355269016 learned_addr learned my addr 172.21.15.3:0/2355269016 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:13.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.575+0000 7f1edebc2700 1 --2- 172.21.15.3:0/2355269016 >> v2:172.21.15.32:3300/0 conn(0x7f1ed80cfd80 0x7f1ed80d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:13.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.579+0000 7f1eddbc0700 1 -- 172.21.15.3:0/2355269016 >> v2:172.21.15.32:3300/0 conn(0x7f1ed80cfd80 msgr2=0x7f1ed80d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:13.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.579+0000 7f1eddbc0700 1 --2- 172.21.15.3:0/2355269016 >> v2:172.21.15.32:3300/0 conn(0x7f1ed80cfd80 0x7f1ed80d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.579+0000 7f1eddbc0700 1 -- 172.21.15.3:0/2355269016 >> v1:172.21.15.3:6789/0 conn(0x7f1ed8156ba0 legacy=0x7f1ed8071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:13.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.579+0000 7f1eddbc0700 1 -- 172.21.15.3:0/2355269016 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed80cd600 con 0x7f1ed815d6e0 2023-03-03T06:26:13.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.579+0000 7f1eddbc0700 1 --2- 172.21.15.3:0/2355269016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ed815d6e0 0x7f1ed80cf840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80b8d60 tx=0x7f1ec8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7268f99cca46ba3 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:13.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.579+0000 7f1edd3bf700 1 -- 172.21.15.3:0/2355269016 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8011830 con 0x7f1ed815d6e0 2023-03-03T06:26:13.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.579+0000 7f1edd3bf700 1 -- 172.21.15.3:0/2355269016 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec8011990 con 0x7f1ed815d6e0 2023-03-03T06:26:13.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.579+0000 7f1edd3bf700 1 -- 172.21.15.3:0/2355269016 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8011d20 con 0x7f1ed815d6e0 2023-03-03T06:26:13.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.579+0000 7f1edf3c3700 1 -- 172.21.15.3:0/2355269016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ed815d6e0 msgr2=0x7f1ed80cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:13.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.579+0000 7f1edf3c3700 1 --2- 172.21.15.3:0/2355269016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ed815d6e0 0x7f1ed80cf840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80b8d60 tx=0x7f1ec8009b30 comp rx=0 tx=0).stop 2023-03-03T06:26:13.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.583+0000 7f1edf3c3700 1 -- 172.21.15.3:0/2355269016 shutdown_connections 2023-03-03T06:26:13.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.583+0000 7f1edf3c3700 1 --2- 172.21.15.3:0/2355269016 >> v2:172.21.15.32:3300/0 conn(0x7f1ed80cfd80 0x7f1ed80d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.583+0000 7f1edf3c3700 1 --2- 172.21.15.3:0/2355269016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ed815d6e0 0x7f1ed80cf840 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.583+0000 7f1edf3c3700 1 -- 172.21.15.3:0/2355269016 >> 172.21.15.3:0/2355269016 conn(0x7f1ed806e5c0 msgr2=0x7f1ed806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:13.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.583+0000 7f1edf3c3700 1 -- 172.21.15.3:0/2355269016 shutdown_connections 2023-03-03T06:26:13.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.583+0000 7f1edf3c3700 1 -- 172.21.15.3:0/2355269016 wait complete. 2023-03-03T06:26:13.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.583+0000 7f1edf3c3700 1 Processor -- start 2023-03-03T06:26:13.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.583+0000 7f1edf3c3700 1 -- start start 2023-03-03T06:26:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1edf3c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ed80cfd80 0x7f1ed814d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1edf3c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed8156ba0 0x7f1ed80b8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1edf3c3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ed815d6e0 0x7f1ed814dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1edf3c3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80b6eb0 con 0x7f1ed80cfd80 2023-03-03T06:26:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1edf3c3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed815ddb0 con 0x7f1ed8156ba0 2023-03-03T06:26:13.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1edf3c3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80b7ca0 con 0x7f1ed815d6e0 2023-03-03T06:26:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1ede3c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ed80cfd80 0x7f1ed814d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:13.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1ede3c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ed80cfd80 0x7f1ed814d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44938/0 (socket says 172.21.15.3:44938) 2023-03-03T06:26:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1ede3c1700 1 -- 172.21.15.3:0/910195716 learned_addr learned my addr 172.21.15.3:0/910195716 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1edebc2700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ed815d6e0 0x7f1ed814dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1eddbc0700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed8156ba0 0x7f1ed80b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1ede3c1700 1 -- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ed815d6e0 msgr2=0x7f1ed814dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1ede3c1700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ed815d6e0 0x7f1ed814dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1ede3c1700 1 -- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed8156ba0 msgr2=0x7f1ed80b8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:13.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1ede3c1700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed8156ba0 0x7f1ed80b8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1ede3c1700 1 -- 172.21.15.3:0/910195716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec80094b0 con 0x7f1ed80cfd80 2023-03-03T06:26:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1eddbc0700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed8156ba0 0x7f1ed80b8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.587+0000 7f1ede3c1700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ed80cfd80 0x7f1ed814d290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400c700 tx=0x7f1ed400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.591+0000 7f1eceffd700 1 -- 172.21.15.3:0/910195716 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4005d90 con 0x7f1ed80cfd80 2023-03-03T06:26:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.591+0000 7f1eceffd700 1 -- 172.21.15.3:0/910195716 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed40070d0 con 0x7f1ed80cfd80 2023-03-03T06:26:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.591+0000 7f1edf3c3700 1 -- 172.21.15.3:0/910195716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed80cef40 con 0x7f1ed80cfd80 2023-03-03T06:26:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.591+0000 7f1eceffd700 1 -- 172.21.15.3:0/910195716 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4007440 con 0x7f1ed80cfd80 2023-03-03T06:26:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.591+0000 7f1edf3c3700 1 -- 172.21.15.3:0/910195716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed80cd8e0 con 0x7f1ed80cfd80 2023-03-03T06:26:13.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.591+0000 7f1edf3c3700 1 -- 172.21.15.3:0/910195716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed80cf0d0 con 0x7f1ed80cfd80 2023-03-03T06:26:13.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.595+0000 7f1eceffd700 1 -- 172.21.15.3:0/910195716 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1ed4019030 con 0x7f1ed80cfd80 2023-03-03T06:26:13.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.595+0000 7f1eceffd700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1ebc064f10 0x7f1ebc0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.599+0000 7f1eceffd700 1 -- 172.21.15.3:0/910195716 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1ed4081560 con 0x7f1ed80cfd80 2023-03-03T06:26:13.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.599+0000 7f1eddbc0700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1ebc064f10 0x7f1ebc0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:13.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.599+0000 7f1eddbc0700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1ebc064f10 0x7f1ebc0673c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8000c00 tx=0x7f1ec8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:13.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.599+0000 7f1eceffd700 1 -- 172.21.15.3:0/910195716 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1ed404a400 con 0x7f1ed80cfd80 2023-03-03T06:26:13.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.775+0000 7f1edf3c3700 1 -- 172.21.15.3:0/910195716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1ed8073e90 con 0x7f1ed80cfd80 2023-03-03T06:26:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.775+0000 7f1eceffd700 1 -- 172.21.15.3:0/910195716 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1ed404f310 con 0x7f1ed80cfd80 2023-03-03T06:26:13.782 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:26:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.779+0000 7f1edf3c3700 1 -- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1ebc064f10 msgr2=0x7f1ebc0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:13.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.779+0000 7f1edf3c3700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1ebc064f10 0x7f1ebc0673c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8000c00 tx=0x7f1ec8008030 comp rx=0 tx=0).stop 2023-03-03T06:26:13.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.779+0000 7f1edf3c3700 1 -- 172.21.15.3:0/910195716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ed80cfd80 msgr2=0x7f1ed814d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:13.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.779+0000 7f1edf3c3700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ed80cfd80 0x7f1ed814d290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400c700 tx=0x7f1ed400ca10 comp rx=0 tx=0).stop 2023-03-03T06:26:13.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.779+0000 7f1edf3c3700 1 -- 172.21.15.3:0/910195716 shutdown_connections 2023-03-03T06:26:13.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.779+0000 7f1edf3c3700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1ebc064f10 0x7f1ebc0673c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.779+0000 7f1edf3c3700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1ed815d6e0 0x7f1ed814dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.779+0000 7f1edf3c3700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ed8156ba0 0x7f1ed80b8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.779+0000 7f1edf3c3700 1 --2- 172.21.15.3:0/910195716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ed80cfd80 0x7f1ed814d290 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:13.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.779+0000 7f1edf3c3700 1 -- 172.21.15.3:0/910195716 >> 172.21.15.3:0/910195716 conn(0x7f1ed806e5c0 msgr2=0x7f1ed80b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.779+0000 7f1edf3c3700 1 -- 172.21.15.3:0/910195716 shutdown_connections 2023-03-03T06:26:13.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:13.779+0000 7f1edf3c3700 1 -- 172.21.15.3:0/910195716 wait complete. 2023-03-03T06:26:13.816 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-03T06:26:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.023+0000 7f08bb169700 1 Processor -- start 2023-03-03T06:26:14.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.023+0000 7f08bb169700 1 -- start start 2023-03-03T06:26:14.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.027+0000 7f08bb169700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f08b415d6e0 0x7f08b40cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.027+0000 7f08bb169700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b40cfd80 0x7f08b40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.027+0000 7f08bb169700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b40cf3b0 con 0x7f08b4156ba0 2023-03-03T06:26:14.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.027+0000 7f08bb169700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b4156f70 con 0x7f08b415d6e0 2023-03-03T06:26:14.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.027+0000 7f08bb169700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b40cd320 con 0x7f08b40cfd80 2023-03-03T06:26:14.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08ba167700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f08b4156ba0 0x7f08b4071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52422/0 (socket says 172.21.15.3:52422) 2023-03-03T06:26:14.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08ba167700 1 -- 172.21.15.3:0/2287452063 learned_addr learned my addr 172.21.15.3:0/2287452063 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:14.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08b9966700 1 --2- 172.21.15.3:0/2287452063 >> v2:172.21.15.3:3300/0 conn(0x7f08b415d6e0 0x7f08b40cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08ba968700 1 --2- 172.21.15.3:0/2287452063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b40cfd80 0x7f08b40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08b9165700 1 -- 172.21.15.3:0/2287452063 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012985651 0 0) 0x7f08b40cf3b0 con 0x7f08b4156ba0 2023-03-03T06:26:14.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08b9165700 1 -- 172.21.15.3:0/2287452063 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08a00035d0 con 0x7f08b4156ba0 2023-03-03T06:26:14.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08ba968700 1 -- 172.21.15.3:0/2287452063 >> v2:172.21.15.3:3300/0 conn(0x7f08b415d6e0 msgr2=0x7f08b40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:14.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08ba968700 1 --2- 172.21.15.3:0/2287452063 >> v2:172.21.15.3:3300/0 conn(0x7f08b415d6e0 0x7f08b40cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08ba968700 1 -- 172.21.15.3:0/2287452063 >> v1:172.21.15.3:6789/0 conn(0x7f08b4156ba0 legacy=0x7f08b4071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08ba968700 1 -- 172.21.15.3:0/2287452063 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b40cd600 con 0x7f08b40cfd80 2023-03-03T06:26:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08b9966700 1 --2- 172.21.15.3:0/2287452063 >> v2:172.21.15.3:3300/0 conn(0x7f08b415d6e0 0x7f08b40cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08ba968700 1 --2- 172.21.15.3:0/2287452063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b40cfd80 0x7f08b40d0b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009850 tx=0x7f08ac009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e573d88c9b534f7 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08b9165700 1 -- 172.21.15.3:0/2287452063 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac011830 con 0x7f08b40cfd80 2023-03-03T06:26:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08b9165700 1 -- 172.21.15.3:0/2287452063 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac011990 con 0x7f08b40cfd80 2023-03-03T06:26:14.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.031+0000 7f08b9165700 1 -- 172.21.15.3:0/2287452063 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac011d00 con 0x7f08b40cfd80 2023-03-03T06:26:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.035+0000 7f08bb169700 1 -- 172.21.15.3:0/2287452063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b40cfd80 msgr2=0x7f08b40d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.035+0000 7f08bb169700 1 --2- 172.21.15.3:0/2287452063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b40cfd80 0x7f08b40d0b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009850 tx=0x7f08ac009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:14.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.035+0000 7f08bb169700 1 -- 172.21.15.3:0/2287452063 shutdown_connections 2023-03-03T06:26:14.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.035+0000 7f08bb169700 1 --2- 172.21.15.3:0/2287452063 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b40cfd80 0x7f08b40d0b30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.035+0000 7f08bb169700 1 --2- 172.21.15.3:0/2287452063 >> v2:172.21.15.3:3300/0 conn(0x7f08b415d6e0 0x7f08b40cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.035+0000 7f08bb169700 1 -- 172.21.15.3:0/2287452063 >> 172.21.15.3:0/2287452063 conn(0x7f08b406e5c0 msgr2=0x7f08b406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:14.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.035+0000 7f08bb169700 1 -- 172.21.15.3:0/2287452063 shutdown_connections 2023-03-03T06:26:14.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.035+0000 7f08bb169700 1 -- 172.21.15.3:0/2287452063 wait complete. 2023-03-03T06:26:14.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.039+0000 7f08bb169700 1 Processor -- start 2023-03-03T06:26:14.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.039+0000 7f08bb169700 1 -- start start 2023-03-03T06:26:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08bb169700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b40cfd80 0x7f08b40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08bb169700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08b4156ba0 0x7f08b40b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08bb169700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b415d6e0 0x7f08b414dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08bb169700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b414e680 con 0x7f08b4156ba0 2023-03-03T06:26:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08bb169700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b40b2bb0 con 0x7f08b40cfd80 2023-03-03T06:26:14.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08bb169700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b414e200 con 0x7f08b415d6e0 2023-03-03T06:26:14.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08b9966700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08b4156ba0 0x7f08b40b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08b9966700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08b4156ba0 0x7f08b40b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44954/0 (socket says 172.21.15.3:44954) 2023-03-03T06:26:14.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08b9966700 1 -- 172.21.15.3:0/3725301331 learned_addr learned my addr 172.21.15.3:0/3725301331 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:14.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08ba968700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b415d6e0 0x7f08b414dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08ba167700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b40cfd80 0x7f08b40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08b9966700 1 -- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b415d6e0 msgr2=0x7f08b414dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08b9966700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b415d6e0 0x7f08b414dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08b9966700 1 -- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b40cfd80 msgr2=0x7f08b40b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08b9966700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b40cfd80 0x7f08b40b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08b9966700 1 -- 172.21.15.3:0/3725301331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a40094b0 con 0x7f08b4156ba0 2023-03-03T06:26:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08b9966700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08b4156ba0 0x7f08b40b2670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f08a400ebe0 tx=0x7f08a400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08aaffd700 1 -- 172.21.15.3:0/3725301331 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400c850 con 0x7f08b4156ba0 2023-03-03T06:26:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08aaffd700 1 -- 172.21.15.3:0/3725301331 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a400c9b0 con 0x7f08b4156ba0 2023-03-03T06:26:14.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08aaffd700 1 -- 172.21.15.3:0/3725301331 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400cd40 con 0x7f08b4156ba0 2023-03-03T06:26:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08bb169700 1 -- 172.21.15.3:0/3725301331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac0094b0 con 0x7f08b4156ba0 2023-03-03T06:26:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.043+0000 7f08bb169700 1 -- 172.21.15.3:0/3725301331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b40cef40 con 0x7f08b4156ba0 2023-03-03T06:26:14.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.047+0000 7f08bb169700 1 -- 172.21.15.3:0/3725301331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b40cd8e0 con 0x7f08b4156ba0 2023-03-03T06:26:14.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.047+0000 7f08aaffd700 1 -- 172.21.15.3:0/3725301331 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f08a4019030 con 0x7f08b4156ba0 2023-03-03T06:26:14.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.051+0000 7f08aaffd700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0898064ee0 0x7f0898067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.051+0000 7f08aaffd700 1 -- 172.21.15.3:0/3725301331 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f08a40821b0 con 0x7f08b4156ba0 2023-03-03T06:26:14.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.051+0000 7f08ba167700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0898064ee0 0x7f0898067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.055+0000 7f08ba167700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0898064ee0 0x7f0898067390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f08b0005cb0 tx=0x7f08b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:14.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.059+0000 7f08aaffd700 1 -- 172.21.15.3:0/3725301331 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f08a404ff30 con 0x7f08b4156ba0 2023-03-03T06:26:14.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.247+0000 7f08bb169700 1 -- 172.21.15.3:0/3725301331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08b40cf0d0 con 0x7f08b4156ba0 2023-03-03T06:26:14.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.247+0000 7f08aaffd700 1 -- 172.21.15.3:0/3725301331 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f08a400cea0 con 0x7f08b4156ba0 2023-03-03T06:26:14.252 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:14.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.251+0000 7f08bb169700 1 -- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0898064ee0 msgr2=0x7f0898067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.251+0000 7f08bb169700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0898064ee0 0x7f0898067390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f08b0005cb0 tx=0x7f08b0009040 comp rx=0 tx=0).stop 2023-03-03T06:26:14.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.251+0000 7f08bb169700 1 -- 172.21.15.3:0/3725301331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08b4156ba0 msgr2=0x7f08b40b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.251+0000 7f08bb169700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08b4156ba0 0x7f08b40b2670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f08a400ebe0 tx=0x7f08a400eef0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.251+0000 7f08bb169700 1 -- 172.21.15.3:0/3725301331 shutdown_connections 2023-03-03T06:26:14.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.251+0000 7f08bb169700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0898064ee0 0x7f0898067390 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.251+0000 7f08bb169700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f08b415d6e0 0x7f08b414dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.251+0000 7f08bb169700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08b4156ba0 0x7f08b40b2670 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.251+0000 7f08bb169700 1 --2- 172.21.15.3:0/3725301331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08b40cfd80 0x7f08b40b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.251+0000 7f08bb169700 1 -- 172.21.15.3:0/3725301331 >> 172.21.15.3:0/3725301331 conn(0x7f08b406e5c0 msgr2=0x7f08b4155e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:14.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.251+0000 7f08bb169700 1 -- 172.21.15.3:0/3725301331 shutdown_connections 2023-03-03T06:26:14.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.251+0000 7f08bb169700 1 -- 172.21.15.3:0/3725301331 wait complete. 2023-03-03T06:26:14.288 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-03T06:26:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.503+0000 7ff40e06f700 1 Processor -- start 2023-03-03T06:26:14.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.503+0000 7ff40e06f700 1 -- start start 2023-03-03T06:26:14.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff40e06f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff40815d6e0 0x7ff4080cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff40e06f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff4080cfd80 0x7ff4080d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff40e06f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4080cf3b0 con 0x7ff408156ba0 2023-03-03T06:26:14.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff40e06f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff408156f70 con 0x7ff4080cfd80 2023-03-03T06:26:14.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff40e06f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080cd320 con 0x7ff40815d6e0 2023-03-03T06:26:14.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff40d06d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff408156ba0 0x7ff408071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52436/0 (socket says 172.21.15.3:52436) 2023-03-03T06:26:14.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff40d06d700 1 -- 172.21.15.3:0/3653075805 learned_addr learned my addr 172.21.15.3:0/3653075805 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:14.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff40d86e700 1 --2- 172.21.15.3:0/3653075805 >> v2:172.21.15.3:3300/0 conn(0x7ff4080cfd80 0x7ff4080d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff40c86c700 1 --2- 172.21.15.3:0/3653075805 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff40815d6e0 0x7ff4080cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff40d86e700 1 -- 172.21.15.3:0/3653075805 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff40815d6e0 msgr2=0x7ff4080cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff40d86e700 1 --2- 172.21.15.3:0/3653075805 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff40815d6e0 0x7ff4080cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff40d86e700 1 -- 172.21.15.3:0/3653075805 >> v1:172.21.15.3:6789/0 conn(0x7ff408156ba0 legacy=0x7ff408071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff3fffff700 1 -- 172.21.15.3:0/3653075805 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885131584 0 0) 0x7ff4080cf3b0 con 0x7ff408156ba0 2023-03-03T06:26:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.507+0000 7ff40d86e700 1 -- 172.21.15.3:0/3653075805 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4080cd600 con 0x7ff4080cfd80 2023-03-03T06:26:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.511+0000 7ff40d86e700 1 --2- 172.21.15.3:0/3653075805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4080cfd80 0x7ff4080d0b30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff400009d90 tx=0x7ff40000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f220681694f74f server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:14.514 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.511+0000 7ff3fffff700 1 -- 172.21.15.3:0/3653075805 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40000e070 con 0x7ff4080cfd80 2023-03-03T06:26:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.511+0000 7ff3fffff700 1 -- 172.21.15.3:0/3653075805 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40000bb20 con 0x7ff4080cfd80 2023-03-03T06:26:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.511+0000 7ff3fffff700 1 -- 172.21.15.3:0/3653075805 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400012430 con 0x7ff4080cfd80 2023-03-03T06:26:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.511+0000 7ff40e06f700 1 -- 172.21.15.3:0/3653075805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4080cfd80 msgr2=0x7ff4080d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.511+0000 7ff40e06f700 1 --2- 172.21.15.3:0/3653075805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4080cfd80 0x7ff4080d0b30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff400009d90 tx=0x7ff40000b330 comp rx=0 tx=0).stop 2023-03-03T06:26:14.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.511+0000 7ff40e06f700 1 -- 172.21.15.3:0/3653075805 shutdown_connections 2023-03-03T06:26:14.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.511+0000 7ff40e06f700 1 --2- 172.21.15.3:0/3653075805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4080cfd80 0x7ff4080d0b30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.511+0000 7ff40e06f700 1 --2- 172.21.15.3:0/3653075805 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff40815d6e0 0x7ff4080cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.511+0000 7ff40e06f700 1 -- 172.21.15.3:0/3653075805 >> 172.21.15.3:0/3653075805 conn(0x7ff40806e5c0 msgr2=0x7ff40806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:14.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.511+0000 7ff40e06f700 1 -- 172.21.15.3:0/3653075805 shutdown_connections 2023-03-03T06:26:14.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.511+0000 7ff40e06f700 1 -- 172.21.15.3:0/3653075805 wait complete. 2023-03-03T06:26:14.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.515+0000 7ff40e06f700 1 Processor -- start 2023-03-03T06:26:14.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.515+0000 7ff40e06f700 1 -- start start 2023-03-03T06:26:14.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.515+0000 7ff40e06f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4080cfd80 0x7ff40814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.515+0000 7ff40e06f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff408156ba0 0x7ff4080b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.515+0000 7ff40e06f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff40815d6e0 0x7ff4080ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.515+0000 7ff40e06f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080b68b0 con 0x7ff4080cfd80 2023-03-03T06:26:14.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.515+0000 7ff40e06f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080b62f0 con 0x7ff40815d6e0 2023-03-03T06:26:14.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.515+0000 7ff40e06f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080b65d0 con 0x7ff408156ba0 2023-03-03T06:26:14.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.515+0000 7ff40d06d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4080cfd80 0x7ff40814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.515+0000 7ff40d06d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4080cfd80 0x7ff40814da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44962/0 (socket says 172.21.15.3:44962) 2023-03-03T06:26:14.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.515+0000 7ff40d06d700 1 -- 172.21.15.3:0/2852388347 learned_addr learned my addr 172.21.15.3:0/2852388347 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:14.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.519+0000 7ff40d86e700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff40815d6e0 0x7ff4080ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.519+0000 7ff40c86c700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff408156ba0 0x7ff4080b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.519+0000 7ff40d06d700 1 -- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff408156ba0 msgr2=0x7ff4080b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.519+0000 7ff40d06d700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff408156ba0 0x7ff4080b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.519+0000 7ff40d06d700 1 -- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff40815d6e0 msgr2=0x7ff4080ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.519+0000 7ff40d06d700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff40815d6e0 0x7ff4080ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.519+0000 7ff40d06d700 1 -- 172.21.15.3:0/2852388347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff400009a70 con 0x7ff4080cfd80 2023-03-03T06:26:14.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.519+0000 7ff40d06d700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4080cfd80 0x7ff40814da00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff40400c580 tx=0x7ff40400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:14.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.519+0000 7ff3fdffb700 1 -- 172.21.15.3:0/2852388347 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404004ca0 con 0x7ff4080cfd80 2023-03-03T06:26:14.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.519+0000 7ff3fdffb700 1 -- 172.21.15.3:0/2852388347 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff404004e20 con 0x7ff4080cfd80 2023-03-03T06:26:14.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.519+0000 7ff3fdffb700 1 -- 172.21.15.3:0/2852388347 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404007310 con 0x7ff4080cfd80 2023-03-03T06:26:14.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.519+0000 7ff40e06f700 1 -- 172.21.15.3:0/2852388347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4080cef40 con 0x7ff4080cfd80 2023-03-03T06:26:14.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.519+0000 7ff40e06f700 1 -- 172.21.15.3:0/2852388347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4080cd8e0 con 0x7ff4080cfd80 2023-03-03T06:26:14.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.523+0000 7ff40e06f700 1 -- 172.21.15.3:0/2852388347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4080cf0d0 con 0x7ff4080cfd80 2023-03-03T06:26:14.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.523+0000 7ff3fdffb700 1 -- 172.21.15.3:0/2852388347 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff40401a020 con 0x7ff4080cfd80 2023-03-03T06:26:14.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.527+0000 7ff3fdffb700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff3ec064ee0 0x7ff3ec067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.527+0000 7ff3fdffb700 1 -- 172.21.15.3:0/2852388347 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff404081f30 con 0x7ff4080cfd80 2023-03-03T06:26:14.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.527+0000 7ff40c86c700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff3ec064ee0 0x7ff3ec067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.527+0000 7ff40c86c700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff3ec064ee0 0x7ff3ec067390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff408156420 tx=0x7ff3f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:14.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.531+0000 7ff3fdffb700 1 -- 172.21.15.3:0/2852388347 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff40404fcb0 con 0x7ff4080cfd80 2023-03-03T06:26:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.711+0000 7ff40e06f700 1 -- 172.21.15.3:0/2852388347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4080b5220 con 0x7ff4080cfd80 2023-03-03T06:26:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.715+0000 7ff3fdffb700 1 -- 172.21.15.3:0/2852388347 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4040532d0 con 0x7ff4080cfd80 2023-03-03T06:26:14.718 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-03-03T06:26:14.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.719+0000 7ff40e06f700 1 -- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff3ec064ee0 msgr2=0x7ff3ec067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.719+0000 7ff40e06f700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff3ec064ee0 0x7ff3ec067390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff408156420 tx=0x7ff3f8008040 comp rx=0 tx=0).stop 2023-03-03T06:26:14.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.719+0000 7ff40e06f700 1 -- 172.21.15.3:0/2852388347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4080cfd80 msgr2=0x7ff40814da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.719+0000 7ff40e06f700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4080cfd80 0x7ff40814da00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff40400c580 tx=0x7ff40400c940 comp rx=0 tx=0).stop 2023-03-03T06:26:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.719+0000 7ff40e06f700 1 -- 172.21.15.3:0/2852388347 shutdown_connections 2023-03-03T06:26:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.719+0000 7ff40e06f700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff3ec064ee0 0x7ff3ec067390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.719+0000 7ff40e06f700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff40815d6e0 0x7ff4080ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.719+0000 7ff40e06f700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff408156ba0 0x7ff4080b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.719+0000 7ff40e06f700 1 --2- 172.21.15.3:0/2852388347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4080cfd80 0x7ff40814da00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.719+0000 7ff40e06f700 1 -- 172.21.15.3:0/2852388347 >> 172.21.15.3:0/2852388347 conn(0x7ff40806e5c0 msgr2=0x7ff4080b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.719+0000 7ff40e06f700 1 -- 172.21.15.3:0/2852388347 shutdown_connections 2023-03-03T06:26:14.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.719+0000 7ff40e06f700 1 -- 172.21.15.3:0/2852388347 wait complete. 2023-03-03T06:26:14.754 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-03T06:26:14.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.963+0000 7f85b6fef700 1 Processor -- start 2023-03-03T06:26:14.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.963+0000 7f85b6fef700 1 -- start start 2023-03-03T06:26:14.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.967+0000 7f85b6fef700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85b0156ba0 0x7f85b0071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.967+0000 7f85b6fef700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b00cf3b0 con 0x7f85b015d6e0 2023-03-03T06:26:14.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.967+0000 7f85b6fef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b0156f70 con 0x7f85b00d0b30 2023-03-03T06:26:14.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.967+0000 7f85b6fef700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b00cd320 con 0x7f85b0156ba0 2023-03-03T06:26:14.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.967+0000 7f85b57ec700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f85b015d6e0 0x7f85b00cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52448/0 (socket says 172.21.15.3:52448) 2023-03-03T06:26:14.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.967+0000 7f85b57ec700 1 -- 172.21.15.3:0/655147042 learned_addr learned my addr 172.21.15.3:0/655147042 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:14.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b5fed700 1 --2- 172.21.15.3:0/655147042 >> v2:172.21.15.3:3300/0 conn(0x7f85b0156ba0 0x7f85b0071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560894501 0 0) 0x7f85b00cf3b0 con 0x7f85b015d6e0 2023-03-03T06:26:14.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f859c0035d0 con 0x7f85b015d6e0 2023-03-03T06:26:14.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4229017887 0 0) 0x7f859c0035d0 con 0x7f85b015d6e0 2023-03-03T06:26:14.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 >> v2:172.21.15.3:3300/0 conn(0x7f85b0156ba0 msgr2=0x7f85b0071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:14.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 --2- 172.21.15.3:0/655147042 >> v2:172.21.15.3:3300/0 conn(0x7f85b0156ba0 0x7f85b0071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 >> v1:172.21.15.32:6789/0 conn(0x7f85b00d0b30 legacy=0x7f85b00b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b00cd600 con 0x7f85b015d6e0 2023-03-03T06:26:14.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f85a0001670 con 0x7f85b015d6e0 2023-03-03T06:26:14.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 >> v1:172.21.15.3:6789/0 conn(0x7f85b015d6e0 legacy=0x7f85b00cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 --2- 172.21.15.3:0/655147042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f859c003a20 0x7f859c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 --2- 172.21.15.3:0/655147042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c004410 0x7f859c0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 --2- 172.21.15.3:0/655147042 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f859c0050d0 0x7f859c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00cd320 con 0x7f859c004410 2023-03-03T06:26:14.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c00c020 con 0x7f859c003a20 2023-03-03T06:26:14.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859c00c1e0 con 0x7f859c0050d0 2023-03-03T06:26:14.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b57ec700 1 --2- 172.21.15.3:0/655147042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c004410 0x7f859c0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b5fed700 1 --2- 172.21.15.3:0/655147042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f859c003a20 0x7f859c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b67ee700 1 --2- 172.21.15.3:0/655147042 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f859c0050d0 0x7f859c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b57ec700 1 -- 172.21.15.3:0/655147042 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f859c0050d0 msgr2=0x7f859c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:14.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b57ec700 1 --2- 172.21.15.3:0/655147042 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f859c0050d0 0x7f859c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b57ec700 1 -- 172.21.15.3:0/655147042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f859c003a20 msgr2=0x7f859c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:14.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b57ec700 1 --2- 172.21.15.3:0/655147042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f859c003a20 0x7f859c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b57ec700 1 -- 172.21.15.3:0/655147042 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c0035d0 con 0x7f859c004410 2023-03-03T06:26:14.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b5fed700 1 --2- 172.21.15.3:0/655147042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f859c003a20 0x7f859c003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:14.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b67ee700 1 -- 172.21.15.3:0/655147042 reap_dead start 2023-03-03T06:26:14.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.971+0000 7f85b57ec700 1 --2- 172.21.15.3:0/655147042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c004410 0x7f859c0048d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f85a000c240 tx=0x7f859c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5399c41659cb8e5b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:14.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.975+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a000cd80 con 0x7f859c004410 2023-03-03T06:26:14.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.975+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a0005980 con 0x7f859c004410 2023-03-03T06:26:14.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.975+0000 7f85b4feb700 1 -- 172.21.15.3:0/655147042 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0005d10 con 0x7f859c004410 2023-03-03T06:26:14.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.975+0000 7f85b6fef700 1 -- 172.21.15.3:0/655147042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c004410 msgr2=0x7f859c0048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.975+0000 7f85b6fef700 1 --2- 172.21.15.3:0/655147042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c004410 0x7f859c0048d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f85a000c240 tx=0x7f859c0058a0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.975+0000 7f85b6fef700 1 -- 172.21.15.3:0/655147042 shutdown_connections 2023-03-03T06:26:14.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.975+0000 7f85b6fef700 1 --2- 172.21.15.3:0/655147042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c004410 0x7f859c0048d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.975+0000 7f85b6fef700 1 -- 172.21.15.3:0/655147042 >> 172.21.15.3:0/655147042 conn(0x7f85b006e5c0 msgr2=0x7f85b006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:14.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.975+0000 7f85b6fef700 1 -- 172.21.15.3:0/655147042 shutdown_connections 2023-03-03T06:26:14.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.975+0000 7f85b6fef700 1 -- 172.21.15.3:0/655147042 wait complete. 2023-03-03T06:26:14.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.979+0000 7f85b6fef700 1 Processor -- start 2023-03-03T06:26:14.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.979+0000 7f85b6fef700 1 -- start start 2023-03-03T06:26:14.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.979+0000 7f85b6fef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f859c004410 0x7f85b01f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.979+0000 7f85b6fef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b00c6260 0x7f85b00c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.979+0000 7f85b6fef700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85b00c8d60 0x7f85b00b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.979+0000 7f85b6fef700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00c8390 con 0x7f85b00c8d60 2023-03-03T06:26:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.979+0000 7f85b6fef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00b3120 con 0x7f85b00c6260 2023-03-03T06:26:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.979+0000 7f85b6fef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00b3390 con 0x7f859c004410 2023-03-03T06:26:14.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.979+0000 7f85b67ee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85b00c8d60 0x7f85b00b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85b5fed700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f859c004410 0x7f85b01f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85b67ee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85b00c8d60 0x7f85b00b4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44998/0 (socket says 172.21.15.3:44998) 2023-03-03T06:26:14.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85b67ee700 1 -- 172.21.15.3:0/2527713794 learned_addr learned my addr 172.21.15.3:0/2527713794 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:14.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85b57ec700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b00c6260 0x7f85b00c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85b5fed700 1 -- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b00c6260 msgr2=0x7f85b00c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85b5fed700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b00c6260 0x7f85b00c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85b5fed700 1 -- 172.21.15.3:0/2527713794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85b00c8d60 msgr2=0x7f85b00b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:14.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85b5fed700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85b00c8d60 0x7f85b00b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:14.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85b5fed700 1 -- 172.21.15.3:0/2527713794 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b00cef40 con 0x7f859c004410 2023-03-03T06:26:14.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85b67ee700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85b00c8d60 0x7f85b00b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:14.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85b5fed700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f859c004410 0x7f85b01f55d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85b00d3d10 tx=0x7f85ac0104a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:14.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85a67fc700 1 -- 172.21.15.3:0/2527713794 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00e050 con 0x7f859c004410 2023-03-03T06:26:14.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85a67fc700 1 -- 172.21.15.3:0/2527713794 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac00f040 con 0x7f859c004410 2023-03-03T06:26:14.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85a67fc700 1 -- 172.21.15.3:0/2527713794 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac010db0 con 0x7f859c004410 2023-03-03T06:26:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85b6fef700 1 -- 172.21.15.3:0/2527713794 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b00cd8e0 con 0x7f859c004410 2023-03-03T06:26:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.983+0000 7f85b6fef700 1 -- 172.21.15.3:0/2527713794 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b00cf0d0 con 0x7f859c004410 2023-03-03T06:26:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.987+0000 7f85b6fef700 1 -- 172.21.15.3:0/2527713794 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b00b3880 con 0x7f859c004410 2023-03-03T06:26:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.987+0000 7f85a67fc700 1 -- 172.21.15.3:0/2527713794 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f85ac003a40 con 0x7f859c004410 2023-03-03T06:26:14.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.991+0000 7f85a67fc700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8594064f30 0x7f85940673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:14.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.991+0000 7f85a67fc700 1 -- 172.21.15.3:0/2527713794 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f85ac083500 con 0x7f859c004410 2023-03-03T06:26:14.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.991+0000 7f85b57ec700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8594064f30 0x7f85940673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:14.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.991+0000 7f85b57ec700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8594064f30 0x7f85940673e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f85a00018b0 tx=0x7f85a0007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:15.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:14.995+0000 7f85a67fc700 1 -- 172.21.15.3:0/2527713794 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f85ac051300 con 0x7f859c004410 2023-03-03T06:26:15.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.171+0000 7f85b6fef700 1 -- 172.21.15.3:0/2527713794 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f85b0073e90 con 0x7f859c004410 2023-03-03T06:26:15.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.171+0000 7f85a67fc700 1 -- 172.21.15.3:0/2527713794 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f85ac054920 con 0x7f859c004410 2023-03-03T06:26:15.178 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:15.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.175+0000 7f85b6fef700 1 -- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8594064f30 msgr2=0x7f85940673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:15.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.175+0000 7f85b6fef700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8594064f30 0x7f85940673e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f85a00018b0 tx=0x7f85a0007000 comp rx=0 tx=0).stop 2023-03-03T06:26:15.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.175+0000 7f85b6fef700 1 -- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f859c004410 msgr2=0x7f85b01f55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:15.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.175+0000 7f85b6fef700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f859c004410 0x7f85b01f55d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85b00d3d10 tx=0x7f85ac0104a0 comp rx=0 tx=0).stop 2023-03-03T06:26:15.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.175+0000 7f85b6fef700 1 -- 172.21.15.3:0/2527713794 shutdown_connections 2023-03-03T06:26:15.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.175+0000 7f85b6fef700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8594064f30 0x7f85940673e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.175+0000 7f85b6fef700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85b00c8d60 0x7f85b00b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.175+0000 7f85b6fef700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85b00c6260 0x7f85b00c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.175+0000 7f85b6fef700 1 --2- 172.21.15.3:0/2527713794 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f859c004410 0x7f85b01f55d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.175+0000 7f85b6fef700 1 -- 172.21.15.3:0/2527713794 >> 172.21.15.3:0/2527713794 conn(0x7f85b006e5c0 msgr2=0x7f85b015a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.175+0000 7f85b6fef700 1 -- 172.21.15.3:0/2527713794 shutdown_connections 2023-03-03T06:26:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.175+0000 7f85b6fef700 1 -- 172.21.15.3:0/2527713794 wait complete. 2023-03-03T06:26:15.245 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-03-03T06:26:15.246 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-03-03T06:26:15.246 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-03-03T06:26:15.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.451+0000 7feb713c6700 1 Processor -- start 2023-03-03T06:26:15.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.455+0000 7feb713c6700 1 -- start start 2023-03-03T06:26:15.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.455+0000 7feb713c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb6c0aff60 0x7feb6c0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:15.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.455+0000 7feb713c6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6c0cbda0 con 0x7feb6c15faf0 2023-03-03T06:26:15.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.455+0000 7feb713c6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6c0d03c0 con 0x7feb6c0b24e0 2023-03-03T06:26:15.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.455+0000 7feb713c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c0cdc90 con 0x7feb6c0aff60 2023-03-03T06:26:15.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.455+0000 7feb6affd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feb6c15faf0 0x7feb6c0cc230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52450/0 (socket says 172.21.15.3:52450) 2023-03-03T06:26:15.462 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.455+0000 7feb6affd700 1 -- 172.21.15.3:0/1114517985 learned_addr learned my addr 172.21.15.3:0/1114517985 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:15.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.455+0000 7feb6a7fc700 1 --2- 172.21.15.3:0/1114517985 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb6c0aff60 0x7feb6c0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:15.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb6a7fc700 1 -- 172.21.15.3:0/1114517985 >> v1:172.21.15.32:6789/0 conn(0x7feb6c0b24e0 legacy=0x7feb6c0b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:15.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb6a7fc700 1 -- 172.21.15.3:0/1114517985 >> v1:172.21.15.3:6789/0 conn(0x7feb6c15faf0 legacy=0x7feb6c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:15.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb6a7fc700 1 -- 172.21.15.3:0/1114517985 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6c0cdf70 con 0x7feb6c0aff60 2023-03-03T06:26:15.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb69ffb700 1 -- 172.21.15.3:0/1114517985 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235778207 0 0) 0x7feb6c0cbda0 con 0x7feb6c15faf0 2023-03-03T06:26:15.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb6a7fc700 1 --2- 172.21.15.3:0/1114517985 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb6c0aff60 0x7feb6c0cfa40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feb60009880 tx=0x7feb60009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5252d1682d1a1b1a server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:15.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb69ffb700 1 -- 172.21.15.3:0/1114517985 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60011830 con 0x7feb6c0aff60 2023-03-03T06:26:15.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb69ffb700 1 -- 172.21.15.3:0/1114517985 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb60011990 con 0x7feb6c0aff60 2023-03-03T06:26:15.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb69ffb700 1 -- 172.21.15.3:0/1114517985 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60011d20 con 0x7feb6c0aff60 2023-03-03T06:26:15.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb713c6700 1 -- 172.21.15.3:0/1114517985 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb6c0aff60 msgr2=0x7feb6c0cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:15.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb713c6700 1 --2- 172.21.15.3:0/1114517985 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb6c0aff60 0x7feb6c0cfa40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feb60009880 tx=0x7feb60009b90 comp rx=0 tx=0).stop 2023-03-03T06:26:15.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb713c6700 1 -- 172.21.15.3:0/1114517985 shutdown_connections 2023-03-03T06:26:15.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb713c6700 1 --2- 172.21.15.3:0/1114517985 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb6c0aff60 0x7feb6c0cfa40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:15.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb713c6700 1 -- 172.21.15.3:0/1114517985 >> 172.21.15.3:0/1114517985 conn(0x7feb6c0d16f0 msgr2=0x7feb6c0cd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:15.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb713c6700 1 -- 172.21.15.3:0/1114517985 shutdown_connections 2023-03-03T06:26:15.467 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.459+0000 7feb713c6700 1 -- 172.21.15.3:0/1114517985 wait complete. 2023-03-03T06:26:15.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.463+0000 7feb713c6700 1 Processor -- start 2023-03-03T06:26:15.469 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb713c6700 1 -- start start 2023-03-03T06:26:15.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb713c6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb6c0aff60 0x7feb6c151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:15.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb713c6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb6c0b24e0 0x7feb6c15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:15.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb713c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb6c15faf0 0x7feb6c15df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:15.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb713c6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c0b1f90 con 0x7feb6c0aff60 2023-03-03T06:26:15.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb713c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c0b13a0 con 0x7feb6c15faf0 2023-03-03T06:26:15.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb713c6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c0b15f0 con 0x7feb6c0b24e0 2023-03-03T06:26:15.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb6affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb6c0aff60 0x7feb6c151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:15.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb6affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb6c0aff60 0x7feb6c151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45004/0 (socket says 172.21.15.3:45004) 2023-03-03T06:26:15.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb6affd700 1 -- 172.21.15.3:0/4039992795 learned_addr learned my addr 172.21.15.3:0/4039992795 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:15.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb6a7fc700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb6c0b24e0 0x7feb6c15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:15.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb6bfff700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb6c15faf0 0x7feb6c15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb6affd700 1 -- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb6c0b24e0 msgr2=0x7feb6c15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb6affd700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb6c0b24e0 0x7feb6c15b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb6affd700 1 -- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb6c15faf0 msgr2=0x7feb6c15df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb6affd700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb6c15faf0 0x7feb6c15df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb6affd700 1 -- 172.21.15.3:0/4039992795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600094b0 con 0x7feb6c0aff60 2023-03-03T06:26:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb6bfff700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb6c15faf0 0x7feb6c15df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:15.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.467+0000 7feb6affd700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb6c0aff60 0x7feb6c151c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00c4e0 tx=0x7feb5c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:15.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.471+0000 7feb537fe700 1 -- 172.21.15.3:0/4039992795 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c004ca0 con 0x7feb6c0aff60 2023-03-03T06:26:15.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.471+0000 7feb537fe700 1 -- 172.21.15.3:0/4039992795 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c004e20 con 0x7feb6c0aff60 2023-03-03T06:26:15.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.471+0000 7feb537fe700 1 -- 172.21.15.3:0/4039992795 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c007310 con 0x7feb6c0aff60 2023-03-03T06:26:15.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.471+0000 7feb713c6700 1 -- 172.21.15.3:0/4039992795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb6c15fec0 con 0x7feb6c0aff60 2023-03-03T06:26:15.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.471+0000 7feb713c6700 1 -- 172.21.15.3:0/4039992795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6c0ce250 con 0x7feb6c0aff60 2023-03-03T06:26:15.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.471+0000 7feb713c6700 1 -- 172.21.15.3:0/4039992795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb6c0cb9e0 con 0x7feb6c0aff60 2023-03-03T06:26:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.475+0000 7feb537fe700 1 -- 172.21.15.3:0/4039992795 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7feb5c01a020 con 0x7feb6c0aff60 2023-03-03T06:26:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.475+0000 7feb537fe700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7feb58064ee0 0x7feb58067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.475+0000 7feb537fe700 1 -- 172.21.15.3:0/4039992795 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7feb5c081e30 con 0x7feb6c0aff60 2023-03-03T06:26:15.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.479+0000 7feb6a7fc700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7feb58064ee0 0x7feb58067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:15.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.479+0000 7feb6a7fc700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7feb58064ee0 0x7feb58067390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feb600090b0 tx=0x7feb60008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:15.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.483+0000 7feb537fe700 1 -- 172.21.15.3:0/4039992795 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7feb5c04fbb0 con 0x7feb6c0aff60 2023-03-03T06:26:15.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:15.715+0000 7feb713c6700 1 -- 172.21.15.3:0/4039992795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7feb6c0b2200 con 0x7feb6c0aff60 2023-03-03T06:26:16.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.687+0000 7feb537fe700 1 -- 172.21.15.3:0/4039992795 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v83) v1 ==== 161+0+0 (secure 0 0 0) 0x7feb5c0531d0 con 0x7feb6c0aff60 2023-03-03T06:26:16.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.691+0000 7feb713c6700 1 -- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7feb58064ee0 msgr2=0x7feb58067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:16.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.691+0000 7feb713c6700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7feb58064ee0 0x7feb58067390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feb600090b0 tx=0x7feb60008030 comp rx=0 tx=0).stop 2023-03-03T06:26:16.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.691+0000 7feb713c6700 1 -- 172.21.15.3:0/4039992795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb6c0aff60 msgr2=0x7feb6c151c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:16.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.691+0000 7feb713c6700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb6c0aff60 0x7feb6c151c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00c4e0 tx=0x7feb5c00c7f0 comp rx=0 tx=0).stop 2023-03-03T06:26:16.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.695+0000 7feb713c6700 1 -- 172.21.15.3:0/4039992795 shutdown_connections 2023-03-03T06:26:16.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.695+0000 7feb713c6700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7feb58064ee0 0x7feb58067390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:16.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.695+0000 7feb713c6700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb6c15faf0 0x7feb6c15df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:16.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.695+0000 7feb713c6700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feb6c0b24e0 0x7feb6c15b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:16.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.695+0000 7feb713c6700 1 --2- 172.21.15.3:0/4039992795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb6c0aff60 0x7feb6c151c50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:16.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.695+0000 7feb713c6700 1 -- 172.21.15.3:0/4039992795 >> 172.21.15.3:0/4039992795 conn(0x7feb6c0d16f0 msgr2=0x7feb6c0cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:16.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.695+0000 7feb713c6700 1 -- 172.21.15.3:0/4039992795 shutdown_connections 2023-03-03T06:26:16.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.695+0000 7feb713c6700 1 -- 172.21.15.3:0/4039992795 wait complete. 2023-03-03T06:26:16.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-03-03T06:26:16.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.947+0000 7f48453f9700 1 Processor -- start 2023-03-03T06:26:16.952 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.947+0000 7f48453f9700 1 -- start start 2023-03-03T06:26:16.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f48453f9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f484006db10 0x7f48400b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:16.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f48453f9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48400cb190 0x7f484006ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:16.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f48453f9700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48400cedd0 con 0x7f48400cfd10 2023-03-03T06:26:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f48453f9700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484015ce40 con 0x7f48400cb190 2023-03-03T06:26:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f48453f9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484015cb60 con 0x7f484006db10 2023-03-03T06:26:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f483ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48400cb190 0x7f484006ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f483ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48400cb190 0x7f484006ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45006/0 (socket says 172.21.15.3:45006) 2023-03-03T06:26:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f483ffff700 1 -- 172.21.15.3:0/3353392270 learned_addr learned my addr 172.21.15.3:0/3353392270 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:16.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f483e7fc700 1 --1- 172.21.15.3:0/3353392270 >> v1:172.21.15.3:6789/0 conn(0x7f48400cfd10 0x7f484006e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52462/0 (socket says 172.21.15.3:52462) 2023-03-03T06:26:16.956 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f483effd700 1 --2- 172.21.15.3:0/3353392270 >> v2:172.21.15.32:3300/0 conn(0x7f484006db10 0x7f48400b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:16.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f483ffff700 1 -- 172.21.15.3:0/3353392270 >> v2:172.21.15.32:3300/0 conn(0x7f484006db10 msgr2=0x7f48400b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:16.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f483ffff700 1 --2- 172.21.15.3:0/3353392270 >> v2:172.21.15.32:3300/0 conn(0x7f484006db10 0x7f48400b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:16.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f483ffff700 1 -- 172.21.15.3:0/3353392270 >> v1:172.21.15.3:6789/0 conn(0x7f48400cfd10 legacy=0x7f484006e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:16.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f483ffff700 1 -- 172.21.15.3:0/3353392270 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48400ce780 con 0x7f48400cb190 2023-03-03T06:26:16.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f483ffff700 1 --2- 172.21.15.3:0/3353392270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48400cb190 0x7f484006ef20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f484006b090 tx=0x7f483800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55dc8f26af3fb6b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:16.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f483dffb700 1 -- 172.21.15.3:0/3353392270 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483800e070 con 0x7f48400cb190 2023-03-03T06:26:16.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f483dffb700 1 -- 172.21.15.3:0/3353392270 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483800b9c0 con 0x7f48400cb190 2023-03-03T06:26:16.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.951+0000 7f483dffb700 1 -- 172.21.15.3:0/3353392270 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483800bd70 con 0x7f48400cb190 2023-03-03T06:26:16.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.955+0000 7f48453f9700 1 -- 172.21.15.3:0/3353392270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48400cb190 msgr2=0x7f484006ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:16.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.955+0000 7f48453f9700 1 --2- 172.21.15.3:0/3353392270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48400cb190 0x7f484006ef20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f484006b090 tx=0x7f483800b330 comp rx=0 tx=0).stop 2023-03-03T06:26:16.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.955+0000 7f48453f9700 1 -- 172.21.15.3:0/3353392270 shutdown_connections 2023-03-03T06:26:16.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.955+0000 7f48453f9700 1 --2- 172.21.15.3:0/3353392270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48400cb190 0x7f484006ef20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:16.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.955+0000 7f48453f9700 1 --2- 172.21.15.3:0/3353392270 >> v2:172.21.15.32:3300/0 conn(0x7f484006db10 0x7f48400b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:16.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.955+0000 7f48453f9700 1 -- 172.21.15.3:0/3353392270 >> 172.21.15.3:0/3353392270 conn(0x7f48400ccc40 msgr2=0x7f48400cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:16.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.955+0000 7f48453f9700 1 -- 172.21.15.3:0/3353392270 shutdown_connections 2023-03-03T06:26:16.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.955+0000 7f48453f9700 1 -- 172.21.15.3:0/3353392270 wait complete. 2023-03-03T06:26:16.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.955+0000 7f48453f9700 1 Processor -- start 2023-03-03T06:26:16.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f48453f9700 1 -- start start 2023-03-03T06:26:16.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f48453f9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f484006db10 0x7f48400c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:16.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f48453f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48400cb190 0x7f4840158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f48453f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48400cfd10 0x7f48400c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f48453f9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48400c54a0 con 0x7f484006db10 2023-03-03T06:26:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f48453f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48400c6c30 con 0x7f48400cfd10 2023-03-03T06:26:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f48453f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484006f0d0 con 0x7f48400cb190 2023-03-03T06:26:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f483effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f484006db10 0x7f48400c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f483effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f484006db10 0x7f48400c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45016/0 (socket says 172.21.15.3:45016) 2023-03-03T06:26:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f483effd700 1 -- 172.21.15.3:0/1234462257 learned_addr learned my addr 172.21.15.3:0/1234462257 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f483e7fc700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48400cb190 0x7f4840158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f483ffff700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48400cfd10 0x7f48400c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f483effd700 1 -- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48400cb190 msgr2=0x7f4840158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f483effd700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48400cb190 0x7f4840158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f483effd700 1 -- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48400cfd10 msgr2=0x7f48400c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f483effd700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48400cfd10 0x7f48400c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f483effd700 1 -- 172.21.15.3:0/1234462257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4838009b30 con 0x7f484006db10 2023-03-03T06:26:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f483e7fc700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48400cb190 0x7f4840158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:16.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f483effd700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f484006db10 0x7f48400c4f60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f483000e7e0 tx=0x7f483000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:16.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f48277fe700 1 -- 172.21.15.3:0/1234462257 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830013610 con 0x7f484006db10 2023-03-03T06:26:16.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f48277fe700 1 -- 172.21.15.3:0/1234462257 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48300137b0 con 0x7f484006db10 2023-03-03T06:26:16.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.959+0000 7f48453f9700 1 -- 172.21.15.3:0/1234462257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48400cace0 con 0x7f484006db10 2023-03-03T06:26:16.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.963+0000 7f48277fe700 1 -- 172.21.15.3:0/1234462257 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830013b60 con 0x7f484006db10 2023-03-03T06:26:16.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.963+0000 7f48453f9700 1 -- 172.21.15.3:0/1234462257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48400cea60 con 0x7f484006db10 2023-03-03T06:26:16.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.963+0000 7f48453f9700 1 -- 172.21.15.3:0/1234462257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4840158540 con 0x7f484006db10 2023-03-03T06:26:16.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.967+0000 7f48277fe700 1 -- 172.21.15.3:0/1234462257 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4830023070 con 0x7f484006db10 2023-03-03T06:26:16.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.967+0000 7f48277fe700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f482c064f10 0x7f482c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:16.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.967+0000 7f48277fe700 1 -- 172.21.15.3:0/1234462257 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4830081f50 con 0x7f484006db10 2023-03-03T06:26:16.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.967+0000 7f483e7fc700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f482c064f10 0x7f482c0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:16.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.971+0000 7f483e7fc700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f482c064f10 0x7f482c0673c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4834000f60 tx=0x7f4834007df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:16.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:16.975+0000 7f48277fe700 1 -- 172.21.15.3:0/1234462257 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f483004fcd0 con 0x7f484006db10 2023-03-03T06:26:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.155+0000 7f48453f9700 1 -- 172.21.15.3:0/1234462257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f484004e180 con 0x7f484006db10 2023-03-03T06:26:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.167+0000 7f48277fe700 1 -- 172.21.15.3:0/1234462257 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f48300532f0 con 0x7f484006db10 2023-03-03T06:26:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.171+0000 7f48453f9700 1 -- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f482c064f10 msgr2=0x7f482c0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.171+0000 7f48453f9700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f482c064f10 0x7f482c0673c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4834000f60 tx=0x7f4834007df0 comp rx=0 tx=0).stop 2023-03-03T06:26:17.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.171+0000 7f48453f9700 1 -- 172.21.15.3:0/1234462257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f484006db10 msgr2=0x7f48400c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:17.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.171+0000 7f48453f9700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f484006db10 0x7f48400c4f60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f483000e7e0 tx=0x7f483000eba0 comp rx=0 tx=0).stop 2023-03-03T06:26:17.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.171+0000 7f48453f9700 1 -- 172.21.15.3:0/1234462257 shutdown_connections 2023-03-03T06:26:17.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.171+0000 7f48453f9700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f482c064f10 0x7f482c0673c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:17.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.171+0000 7f48453f9700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48400cfd10 0x7f48400c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.171+0000 7f48453f9700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48400cb190 0x7f4840158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.171+0000 7f48453f9700 1 --2- 172.21.15.3:0/1234462257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f484006db10 0x7f48400c4f60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.171+0000 7f48453f9700 1 -- 172.21.15.3:0/1234462257 >> 172.21.15.3:0/1234462257 conn(0x7f48400ccc40 msgr2=0x7f484014d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.171+0000 7f48453f9700 1 -- 172.21.15.3:0/1234462257 shutdown_connections 2023-03-03T06:26:17.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.171+0000 7f48453f9700 1 -- 172.21.15.3:0/1234462257 wait complete. 2023-03-03T06:26:17.208 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-03-03T06:26:17.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-03-03T06:26:17.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-03-03T06:26:17.322 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-03-03T06:26:17.323 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:26:17.323 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:17.344 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:26:17.344 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:17.406 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:26:17.407 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:17.476 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:17.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.727+0000 7f44f8e4c700 1 Processor -- start 2023-03-03T06:26:17.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.727+0000 7f44f8e4c700 1 -- start start 2023-03-03T06:26:17.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.727+0000 7f44f8e4c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44f415dec0 0x7f44f415e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44f8e4c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f40c9a20 con 0x7f44f40adf20 2023-03-03T06:26:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44f8e4c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f40c9fe0 con 0x7f44f40cb180 2023-03-03T06:26:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44f8e4c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f40ae2f0 con 0x7f44f415dec0 2023-03-03T06:26:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44eaffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44f415dec0 0x7f44f415e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44f37fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f44f40adf20 0x7f44f40b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52470/0 (socket says 172.21.15.3:52470) 2023-03-03T06:26:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44f37fe700 1 -- 172.21.15.3:0/3183315456 learned_addr learned my addr 172.21.15.3:0/3183315456 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44eaffd700 1 -- 172.21.15.3:0/3183315456 >> v1:172.21.15.32:6789/0 conn(0x7f44f40cb180 legacy=0x7f44f40ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44eaffd700 1 -- 172.21.15.3:0/3183315456 >> v1:172.21.15.3:6789/0 conn(0x7f44f40adf20 legacy=0x7f44f40b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44eaffd700 1 -- 172.21.15.3:0/3183315456 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f415a7d0 con 0x7f44f415dec0 2023-03-03T06:26:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44f2ffd700 1 -- 172.21.15.3:0/3183315456 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217588903 0 0) 0x7f44f40c9a20 con 0x7f44f40adf20 2023-03-03T06:26:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44eaffd700 1 --2- 172.21.15.3:0/3183315456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f415dec0 0x7f44f415e310 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009850 tx=0x7f44dc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da942355fceaa98e server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44f2ffd700 1 -- 172.21.15.3:0/3183315456 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc011830 con 0x7f44f415dec0 2023-03-03T06:26:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44f2ffd700 1 -- 172.21.15.3:0/3183315456 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44dc011990 con 0x7f44f415dec0 2023-03-03T06:26:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44f2ffd700 1 -- 172.21.15.3:0/3183315456 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc011d20 con 0x7f44f415dec0 2023-03-03T06:26:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3183315456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f415dec0 msgr2=0x7f44f415e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.731+0000 7f44f8e4c700 1 --2- 172.21.15.3:0/3183315456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f415dec0 0x7f44f415e310 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009850 tx=0x7f44dc009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.735+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3183315456 shutdown_connections 2023-03-03T06:26:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.735+0000 7f44f8e4c700 1 --2- 172.21.15.3:0/3183315456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f415dec0 0x7f44f415e310 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.735+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3183315456 >> 172.21.15.3:0/3183315456 conn(0x7f44f4070a30 msgr2=0x7f44f4070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.735+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3183315456 shutdown_connections 2023-03-03T06:26:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.735+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3183315456 wait complete. 2023-03-03T06:26:17.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.739+0000 7f44f8e4c700 1 Processor -- start 2023-03-03T06:26:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.739+0000 7f44f8e4c700 1 -- start start 2023-03-03T06:26:17.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.739+0000 7f44f8e4c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f40adf20 0x7f44f41f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:17.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.739+0000 7f44f8e4c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f40cb180 0x7f44f4156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:17.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.739+0000 7f44f8e4c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f415dec0 0x7f44f4070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:17.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.739+0000 7f44f8e4c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40cc4f0 con 0x7f44f40cb180 2023-03-03T06:26:17.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.739+0000 7f44f8e4c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40cc7d0 con 0x7f44f40adf20 2023-03-03T06:26:17.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.739+0000 7f44f8e4c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f40ccab0 con 0x7f44f415dec0 2023-03-03T06:26:17.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.739+0000 7f44eaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f40cb180 0x7f44f4156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:17.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.739+0000 7f44eaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f40cb180 0x7f44f4156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45042/0 (socket says 172.21.15.3:45042) 2023-03-03T06:26:17.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.739+0000 7f44eaffd700 1 -- 172.21.15.3:0/3199904974 learned_addr learned my addr 172.21.15.3:0/3199904974 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:17.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.739+0000 7f44f3fff700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f415dec0 0x7f44f4070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:17.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.743+0000 7f44f37fe700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f40adf20 0x7f44f41f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:17.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.743+0000 7f44eaffd700 1 -- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f415dec0 msgr2=0x7f44f4070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:17.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.743+0000 7f44eaffd700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f415dec0 0x7f44f4070e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:17.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.743+0000 7f44eaffd700 1 -- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f40adf20 msgr2=0x7f44f41f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:17.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.743+0000 7f44eaffd700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f40adf20 0x7f44f41f5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:17.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.743+0000 7f44eaffd700 1 -- 172.21.15.3:0/3199904974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc0094b0 con 0x7f44f40cb180 2023-03-03T06:26:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.743+0000 7f44eaffd700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f40cb180 0x7f44f4156fd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f44dc014040 tx=0x7f44dc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.743+0000 7f44f0ff9700 1 -- 172.21.15.3:0/3199904974 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc00ba70 con 0x7f44f40cb180 2023-03-03T06:26:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.743+0000 7f44f0ff9700 1 -- 172.21.15.3:0/3199904974 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44dc01e070 con 0x7f44f40cb180 2023-03-03T06:26:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.743+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3199904974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f415ebf0 con 0x7f44f40cb180 2023-03-03T06:26:17.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.743+0000 7f44f0ff9700 1 -- 172.21.15.3:0/3199904974 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc00bdf0 con 0x7f44f40cb180 2023-03-03T06:26:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.743+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3199904974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f415e910 con 0x7f44f40cb180 2023-03-03T06:26:17.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.747+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3199904974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f40c9c70 con 0x7f44f40cb180 2023-03-03T06:26:17.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.747+0000 7f44f0ff9700 1 -- 172.21.15.3:0/3199904974 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f44dc02a020 con 0x7f44f40cb180 2023-03-03T06:26:17.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.747+0000 7f44f0ff9700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f44d4064ee0 0x7f44d4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.747+0000 7f44f0ff9700 1 -- 172.21.15.3:0/3199904974 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f44dc083520 con 0x7f44f40cb180 2023-03-03T06:26:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.751+0000 7f44f37fe700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f44d4064ee0 0x7f44d4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:17.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.751+0000 7f44f37fe700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f44d4064ee0 0x7f44d4067390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f44e4000f60 tx=0x7f44e4007c40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:17.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.755+0000 7f44f0ff9700 1 -- 172.21.15.3:0/3199904974 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f44dc051220 con 0x7f44f40cb180 2023-03-03T06:26:17.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.939+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3199904974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44f4073e90 con 0x7f44f40cb180 2023-03-03T06:26:17.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.943+0000 7f44f0ff9700 1 -- 172.21.15.3:0/3199904974 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f44dc054840 con 0x7f44f40cb180 2023-03-03T06:26:17.946 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:17.947 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:53.877945+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:17.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.943+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f44d4064ee0 msgr2=0x7f44d4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:17.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.943+0000 7f44f8e4c700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f44d4064ee0 0x7f44d4067390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f44e4000f60 tx=0x7f44e4007c40 comp rx=0 tx=0).stop 2023-03-03T06:26:17.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.943+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3199904974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f40cb180 msgr2=0x7f44f4156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:17.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.943+0000 7f44f8e4c700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f40cb180 0x7f44f4156fd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f44dc014040 tx=0x7f44dc004770 comp rx=0 tx=0).stop 2023-03-03T06:26:17.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.943+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3199904974 shutdown_connections 2023-03-03T06:26:17.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.943+0000 7f44f8e4c700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f44d4064ee0 0x7f44d4067390 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:17.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.943+0000 7f44f8e4c700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f44f415dec0 0x7f44f4070e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:17.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.943+0000 7f44f8e4c700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44f40cb180 0x7f44f4156fd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:17.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.943+0000 7f44f8e4c700 1 --2- 172.21.15.3:0/3199904974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f44f40adf20 0x7f44f41f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:17.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.943+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3199904974 >> 172.21.15.3:0/3199904974 conn(0x7f44f4070a30 msgr2=0x7f44f415cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:17.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.947+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3199904974 shutdown_connections 2023-03-03T06:26:17.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:17.947+0000 7f44f8e4c700 1 -- 172.21.15.3:0/3199904974 wait complete. 2023-03-03T06:26:17.986 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-03T06:25:11.468534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-03T06:25:12.813283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-03T06:25:16.382800+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-03T06:26:17.987 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:26:18.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.191+0000 7efc9e123700 1 Processor -- start 2023-03-03T06:26:18.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.191+0000 7efc9e123700 1 -- start start 2023-03-03T06:26:18.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.191+0000 7efc9e123700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efc9815d6e0 0x7efc980cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:18.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.191+0000 7efc9e123700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc980cf3b0 con 0x7efc98156ba0 2023-03-03T06:26:18.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.191+0000 7efc9e123700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc98156f70 con 0x7efc980cfd80 2023-03-03T06:26:18.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.191+0000 7efc9e123700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc980cd320 con 0x7efc9815d6e0 2023-03-03T06:26:18.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc9d121700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efc98156ba0 0x7efc98071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52476/0 (socket says 172.21.15.3:52476) 2023-03-03T06:26:18.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc9d121700 1 -- 172.21.15.3:0/4043092017 learned_addr learned my addr 172.21.15.3:0/4043092017 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:18.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc9c920700 1 --2- 172.21.15.3:0/4043092017 >> v2:172.21.15.3:3300/0 conn(0x7efc9815d6e0 0x7efc980cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:18.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc8ffff700 1 -- 172.21.15.3:0/4043092017 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604924891 0 0) 0x7efc980cf3b0 con 0x7efc98156ba0 2023-03-03T06:26:18.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc8ffff700 1 -- 172.21.15.3:0/4043092017 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc840035d0 con 0x7efc98156ba0 2023-03-03T06:26:18.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc9c920700 1 -- 172.21.15.3:0/4043092017 >> v1:172.21.15.32:6789/0 conn(0x7efc980cfd80 legacy=0x7efc980d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc9c920700 1 -- 172.21.15.3:0/4043092017 >> v1:172.21.15.3:6789/0 conn(0x7efc98156ba0 legacy=0x7efc98071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc9c920700 1 -- 172.21.15.3:0/4043092017 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc980cd600 con 0x7efc9815d6e0 2023-03-03T06:26:18.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc9c920700 1 --2- 172.21.15.3:0/4043092017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc9815d6e0 0x7efc980cf840 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efc88009850 tx=0x7efc88009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb3bf03fab4f65c server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:18.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc8ffff700 1 -- 172.21.15.3:0/4043092017 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88011830 con 0x7efc9815d6e0 2023-03-03T06:26:18.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc8ffff700 1 -- 172.21.15.3:0/4043092017 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc88011990 con 0x7efc9815d6e0 2023-03-03T06:26:18.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc8ffff700 1 -- 172.21.15.3:0/4043092017 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88011d00 con 0x7efc9815d6e0 2023-03-03T06:26:18.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc9e123700 1 -- 172.21.15.3:0/4043092017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc9815d6e0 msgr2=0x7efc980cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.195+0000 7efc9e123700 1 --2- 172.21.15.3:0/4043092017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc9815d6e0 0x7efc980cf840 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efc88009850 tx=0x7efc88009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.199+0000 7efc9e123700 1 -- 172.21.15.3:0/4043092017 shutdown_connections 2023-03-03T06:26:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.199+0000 7efc9e123700 1 --2- 172.21.15.3:0/4043092017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc9815d6e0 0x7efc980cf840 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.199+0000 7efc9e123700 1 -- 172.21.15.3:0/4043092017 >> 172.21.15.3:0/4043092017 conn(0x7efc9806e5c0 msgr2=0x7efc9806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.199+0000 7efc9e123700 1 -- 172.21.15.3:0/4043092017 shutdown_connections 2023-03-03T06:26:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.199+0000 7efc9e123700 1 -- 172.21.15.3:0/4043092017 wait complete. 2023-03-03T06:26:18.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.199+0000 7efc9e123700 1 Processor -- start 2023-03-03T06:26:18.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.199+0000 7efc9e123700 1 -- start start 2023-03-03T06:26:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9e123700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc980cfd80 0x7efc9814e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9e123700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc98156ba0 0x7efc98152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9e123700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc9815d6e0 0x7efc980ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9e123700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc980b6f00 con 0x7efc980cfd80 2023-03-03T06:26:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9e123700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc980b71e0 con 0x7efc98156ba0 2023-03-03T06:26:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9e123700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc980b74c0 con 0x7efc9815d6e0 2023-03-03T06:26:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9d121700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc980cfd80 0x7efc9814e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9d121700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc980cfd80 0x7efc9814e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45060/0 (socket says 172.21.15.3:45060) 2023-03-03T06:26:18.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9d121700 1 -- 172.21.15.3:0/219446831 learned_addr learned my addr 172.21.15.3:0/219446831 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:18.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9c920700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc98156ba0 0x7efc98152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:18.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9d922700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc9815d6e0 0x7efc980ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:18.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9d121700 1 -- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc9815d6e0 msgr2=0x7efc980ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9d121700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc9815d6e0 0x7efc980ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9d121700 1 -- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc98156ba0 msgr2=0x7efc98152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9d121700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc98156ba0 0x7efc98152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9d121700 1 -- 172.21.15.3:0/219446831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc880094b0 con 0x7efc980cfd80 2023-03-03T06:26:18.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9d121700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc980cfd80 0x7efc9814e210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c700 tx=0x7efc9400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc8dffb700 1 -- 172.21.15.3:0/219446831 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94005d90 con 0x7efc980cfd80 2023-03-03T06:26:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc9e123700 1 -- 172.21.15.3:0/219446831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc980cef40 con 0x7efc980cfd80 2023-03-03T06:26:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.203+0000 7efc8dffb700 1 -- 172.21.15.3:0/219446831 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc940070d0 con 0x7efc980cfd80 2023-03-03T06:26:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.207+0000 7efc8dffb700 1 -- 172.21.15.3:0/219446831 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94007480 con 0x7efc980cfd80 2023-03-03T06:26:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.207+0000 7efc9e123700 1 -- 172.21.15.3:0/219446831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc980cd8e0 con 0x7efc980cfd80 2023-03-03T06:26:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.207+0000 7efc9e123700 1 -- 172.21.15.3:0/219446831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc980cf0d0 con 0x7efc980cfd80 2023-03-03T06:26:18.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.211+0000 7efc8dffb700 1 -- 172.21.15.3:0/219446831 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7efc94019030 con 0x7efc980cfd80 2023-03-03T06:26:18.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.211+0000 7efc8dffb700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efc7c064f10 0x7efc7c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.211+0000 7efc8dffb700 1 -- 172.21.15.3:0/219446831 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efc940816e0 con 0x7efc980cfd80 2023-03-03T06:26:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.211+0000 7efc9c920700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efc7c064f10 0x7efc7c0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:18.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.215+0000 7efc9c920700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efc7c064f10 0x7efc7c0673c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efc880090b0 tx=0x7efc8801d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:18.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.219+0000 7efc8dffb700 1 -- 172.21.15.3:0/219446831 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7efc9404f3e0 con 0x7efc980cfd80 2023-03-03T06:26:18.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.395+0000 7efc9e123700 1 -- 172.21.15.3:0/219446831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc98073e90 con 0x7efc980cfd80 2023-03-03T06:26:18.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.395+0000 7efc8dffb700 1 -- 172.21.15.3:0/219446831 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc94052a00 con 0x7efc980cfd80 2023-03-03T06:26:18.400 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:26:18.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.399+0000 7efc9e123700 1 -- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efc7c064f10 msgr2=0x7efc7c0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.399+0000 7efc9e123700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efc7c064f10 0x7efc7c0673c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efc880090b0 tx=0x7efc8801d040 comp rx=0 tx=0).stop 2023-03-03T06:26:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.399+0000 7efc9e123700 1 -- 172.21.15.3:0/219446831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc980cfd80 msgr2=0x7efc9814e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.399+0000 7efc9e123700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc980cfd80 0x7efc9814e210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c700 tx=0x7efc9400ca10 comp rx=0 tx=0).stop 2023-03-03T06:26:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.399+0000 7efc9e123700 1 -- 172.21.15.3:0/219446831 shutdown_connections 2023-03-03T06:26:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.399+0000 7efc9e123700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efc7c064f10 0x7efc7c0673c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.399+0000 7efc9e123700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efc9815d6e0 0x7efc980ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.399+0000 7efc9e123700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efc98156ba0 0x7efc98152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.399+0000 7efc9e123700 1 --2- 172.21.15.3:0/219446831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc980cfd80 0x7efc9814e210 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.399+0000 7efc9e123700 1 -- 172.21.15.3:0/219446831 >> 172.21.15.3:0/219446831 conn(0x7efc9806e5c0 msgr2=0x7efc98150fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:18.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.399+0000 7efc9e123700 1 -- 172.21.15.3:0/219446831 shutdown_connections 2023-03-03T06:26:18.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.399+0000 7efc9e123700 1 -- 172.21.15.3:0/219446831 wait complete. 2023-03-03T06:26:18.438 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:26:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.647+0000 7fa92556b700 1 Processor -- start 2023-03-03T06:26:18.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.647+0000 7fa92556b700 1 -- start start 2023-03-03T06:26:18.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.651+0000 7fa92556b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9200cad60 0x7fa9200b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.651+0000 7fa92556b700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9200cab60 con 0x7fa92006f970 2023-03-03T06:26:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.651+0000 7fa92556b700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92006f4e0 con 0x7fa920158a00 2023-03-03T06:26:18.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.651+0000 7fa92556b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92006ef00 con 0x7fa9200cad60 2023-03-03T06:26:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.651+0000 7fa91e7fc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9200cad60 0x7fa9200b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.651+0000 7fa91effd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa920158a00 0x7fa9200cdd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:54338/0 (socket says 172.21.15.3:54338) 2023-03-03T06:26:18.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.651+0000 7fa91effd700 1 -- 172.21.15.3:0/1432278309 learned_addr learned my addr 172.21.15.3:0/1432278309 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:18.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa91dffb700 1 -- 172.21.15.3:0/1432278309 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739239453 0 0) 0x7fa92006f4e0 con 0x7fa920158a00 2023-03-03T06:26:18.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa91dffb700 1 -- 172.21.15.3:0/1432278309 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9080035d0 con 0x7fa920158a00 2023-03-03T06:26:18.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa91e7fc700 1 -- 172.21.15.3:0/1432278309 >> v1:172.21.15.32:6789/0 conn(0x7fa920158a00 legacy=0x7fa9200cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa91e7fc700 1 -- 172.21.15.3:0/1432278309 >> v1:172.21.15.3:6789/0 conn(0x7fa92006f970 legacy=0x7fa92006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa91e7fc700 1 -- 172.21.15.3:0/1432278309 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa920070870 con 0x7fa9200cad60 2023-03-03T06:26:18.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa91e7fc700 1 --2- 172.21.15.3:0/1432278309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9200cad60 0x7fa9200b2540 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa91400a9f0 tx=0x7fa91400ad00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bc6c1641c6b673b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa91dffb700 1 -- 172.21.15.3:0/1432278309 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91400cb70 con 0x7fa9200cad60 2023-03-03T06:26:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa91dffb700 1 -- 172.21.15.3:0/1432278309 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91400ae90 con 0x7fa9200cad60 2023-03-03T06:26:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa91dffb700 1 -- 172.21.15.3:0/1432278309 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914018400 con 0x7fa9200cad60 2023-03-03T06:26:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa92556b700 1 -- 172.21.15.3:0/1432278309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9200cad60 msgr2=0x7fa9200b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa92556b700 1 --2- 172.21.15.3:0/1432278309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9200cad60 0x7fa9200b2540 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa91400a9f0 tx=0x7fa91400ad00 comp rx=0 tx=0).stop 2023-03-03T06:26:18.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa92556b700 1 -- 172.21.15.3:0/1432278309 shutdown_connections 2023-03-03T06:26:18.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa92556b700 1 --2- 172.21.15.3:0/1432278309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9200cad60 0x7fa9200b2540 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa92556b700 1 -- 172.21.15.3:0/1432278309 >> 172.21.15.3:0/1432278309 conn(0x7fa9200ccc40 msgr2=0x7fa9200cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:18.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.655+0000 7fa92556b700 1 -- 172.21.15.3:0/1432278309 shutdown_connections 2023-03-03T06:26:18.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.659+0000 7fa92556b700 1 -- 172.21.15.3:0/1432278309 wait complete. 2023-03-03T06:26:18.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa92556b700 1 Processor -- start 2023-03-03T06:26:18.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa92556b700 1 -- start start 2023-03-03T06:26:18.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa92556b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92006f970 0x7fa9200c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:18.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa92556b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9200cad60 0x7fa9200cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:18.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa92556b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa920158a00 0x7fa9200cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:18.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa92556b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9200c99c0 con 0x7fa92006f970 2023-03-03T06:26:18.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa92556b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9201f65c0 con 0x7fa9200cad60 2023-03-03T06:26:18.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa92556b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9201f68a0 con 0x7fa920158a00 2023-03-03T06:26:18.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa91effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92006f970 0x7fa9200c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:18.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa91effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92006f970 0x7fa9200c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45068/0 (socket says 172.21.15.3:45068) 2023-03-03T06:26:18.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa91effd700 1 -- 172.21.15.3:0/1491856557 learned_addr learned my addr 172.21.15.3:0/1491856557 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:18.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa91ffff700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa920158a00 0x7fa9200cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:18.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa91e7fc700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9200cad60 0x7fa9200cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:18.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa91effd700 1 -- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa920158a00 msgr2=0x7fa9200cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.663+0000 7fa91effd700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa920158a00 0x7fa9200cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.667+0000 7fa91effd700 1 -- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9200cad60 msgr2=0x7fa9200cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.667+0000 7fa91effd700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9200cad60 0x7fa9200cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.667+0000 7fa91effd700 1 -- 172.21.15.3:0/1491856557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91400a650 con 0x7fa92006f970 2023-03-03T06:26:18.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.667+0000 7fa91e7fc700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9200cad60 0x7fa9200cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:18.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.667+0000 7fa91effd700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92006f970 0x7fa9200c8f40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa910001260 tx=0x7fa91000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:18.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.667+0000 7fa9077fe700 1 -- 172.21.15.3:0/1491856557 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9100074e0 con 0x7fa92006f970 2023-03-03T06:26:18.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.667+0000 7fa9077fe700 1 -- 172.21.15.3:0/1491856557 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa910007640 con 0x7fa92006f970 2023-03-03T06:26:18.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.667+0000 7fa92556b700 1 -- 172.21.15.3:0/1491856557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9200ce4a0 con 0x7fa92006f970 2023-03-03T06:26:18.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.667+0000 7fa9077fe700 1 -- 172.21.15.3:0/1491856557 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910005a50 con 0x7fa92006f970 2023-03-03T06:26:18.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.667+0000 7fa92556b700 1 -- 172.21.15.3:0/1491856557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92006f170 con 0x7fa92006f970 2023-03-03T06:26:18.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.667+0000 7fa92556b700 1 -- 172.21.15.3:0/1491856557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa920158dd0 con 0x7fa92006f970 2023-03-03T06:26:18.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.671+0000 7fa9077fe700 1 -- 172.21.15.3:0/1491856557 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa910019030 con 0x7fa92006f970 2023-03-03T06:26:18.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.671+0000 7fa9077fe700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa90c064ee0 0x7fa90c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:18.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.671+0000 7fa9077fe700 1 -- 172.21.15.3:0/1491856557 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa9100814a0 con 0x7fa92006f970 2023-03-03T06:26:18.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.675+0000 7fa91e7fc700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa90c064ee0 0x7fa90c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:18.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.675+0000 7fa91e7fc700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa90c064ee0 0x7fa90c067390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa91400a1c0 tx=0x7fa91401b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:18.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.679+0000 7fa9077fe700 1 -- 172.21.15.3:0/1491856557 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa91004a400 con 0x7fa92006f970 2023-03-03T06:26:18.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.851+0000 7fa92556b700 1 -- 172.21.15.3:0/1491856557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa92004e180 con 0x7fa92006f970 2023-03-03T06:26:18.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.855+0000 7fa9077fe700 1 -- 172.21.15.3:0/1491856557 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa910017020 con 0x7fa92006f970 2023-03-03T06:26:18.858 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:26:18.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.859+0000 7fa92556b700 1 -- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa90c064ee0 msgr2=0x7fa90c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.859+0000 7fa92556b700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa90c064ee0 0x7fa90c067390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa91400a1c0 tx=0x7fa91401b040 comp rx=0 tx=0).stop 2023-03-03T06:26:18.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.859+0000 7fa92556b700 1 -- 172.21.15.3:0/1491856557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92006f970 msgr2=0x7fa9200c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:18.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.859+0000 7fa92556b700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92006f970 0x7fa9200c8f40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa910001260 tx=0x7fa91000c9e0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.859+0000 7fa92556b700 1 -- 172.21.15.3:0/1491856557 shutdown_connections 2023-03-03T06:26:18.862 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.859+0000 7fa92556b700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa90c064ee0 0x7fa90c067390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.859+0000 7fa92556b700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa920158a00 0x7fa9200cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.859+0000 7fa92556b700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9200cad60 0x7fa9200cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.859+0000 7fa92556b700 1 --2- 172.21.15.3:0/1491856557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92006f970 0x7fa9200c8f40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:18.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.859+0000 7fa92556b700 1 -- 172.21.15.3:0/1491856557 >> 172.21.15.3:0/1491856557 conn(0x7fa9200ccc40 msgr2=0x7fa92014cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:18.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.859+0000 7fa92556b700 1 -- 172.21.15.3:0/1491856557 shutdown_connections 2023-03-03T06:26:18.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:18.859+0000 7fa92556b700 1 -- 172.21.15.3:0/1491856557 wait complete. 2023-03-03T06:26:18.896 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-03T06:26:19.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.103+0000 7f3824e6d700 1 Processor -- start 2023-03-03T06:26:19.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.103+0000 7f3824e6d700 1 -- start start 2023-03-03T06:26:19.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.107+0000 7f3824e6d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3820156ba0 0x7f3820071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.107+0000 7f3824e6d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38200cf3b0 con 0x7f382015d6e0 2023-03-03T06:26:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.107+0000 7f3824e6d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3820156f70 con 0x7f38200d0b30 2023-03-03T06:26:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.107+0000 7f3824e6d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38200cd320 con 0x7f3820156ba0 2023-03-03T06:26:19.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.107+0000 7f381f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3820156ba0 0x7f3820071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.107+0000 7f381f7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3820156ba0 0x7f3820071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45084/0 (socket says 172.21.15.3:45084) 2023-03-03T06:26:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.107+0000 7f381f7fe700 1 -- 172.21.15.3:0/2906684541 learned_addr learned my addr 172.21.15.3:0/2906684541 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:19.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.107+0000 7f381effd700 1 --1- 172.21.15.3:0/2906684541 >> v1:172.21.15.3:6789/0 conn(0x7f382015d6e0 0x7f38200cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52482/0 (socket says 172.21.15.3:52482) 2023-03-03T06:26:19.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.107+0000 7f381f7fe700 1 -- 172.21.15.3:0/2906684541 >> v1:172.21.15.32:6789/0 conn(0x7f38200d0b30 legacy=0x7f38200b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.107+0000 7f381f7fe700 1 -- 172.21.15.3:0/2906684541 >> v1:172.21.15.3:6789/0 conn(0x7f382015d6e0 legacy=0x7f38200cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.107+0000 7f381f7fe700 1 -- 172.21.15.3:0/2906684541 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38200cd600 con 0x7f3820156ba0 2023-03-03T06:26:19.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.107+0000 7f381e7fc700 1 -- 172.21.15.3:0/2906684541 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219786907 0 0) 0x7f38200cf3b0 con 0x7f382015d6e0 2023-03-03T06:26:19.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.111+0000 7f381f7fe700 1 --2- 172.21.15.3:0/2906684541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3820156ba0 0x7f3820071280 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3810009910 tx=0x7f3810009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54763df96be5f3db server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:19.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.111+0000 7f381e7fc700 1 -- 172.21.15.3:0/2906684541 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381000ba60 con 0x7f3820156ba0 2023-03-03T06:26:19.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.111+0000 7f381e7fc700 1 -- 172.21.15.3:0/2906684541 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381000bbc0 con 0x7f3820156ba0 2023-03-03T06:26:19.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.111+0000 7f381e7fc700 1 -- 172.21.15.3:0/2906684541 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810009e60 con 0x7f3820156ba0 2023-03-03T06:26:19.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.111+0000 7f3824e6d700 1 -- 172.21.15.3:0/2906684541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3820156ba0 msgr2=0x7f3820071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.111+0000 7f3824e6d700 1 --2- 172.21.15.3:0/2906684541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3820156ba0 0x7f3820071280 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3810009910 tx=0x7f3810009c20 comp rx=0 tx=0).stop 2023-03-03T06:26:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.111+0000 7f3824e6d700 1 -- 172.21.15.3:0/2906684541 shutdown_connections 2023-03-03T06:26:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.111+0000 7f3824e6d700 1 --2- 172.21.15.3:0/2906684541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3820156ba0 0x7f3820071280 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.111+0000 7f3824e6d700 1 -- 172.21.15.3:0/2906684541 >> 172.21.15.3:0/2906684541 conn(0x7f382006e5c0 msgr2=0x7f382006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.111+0000 7f3824e6d700 1 -- 172.21.15.3:0/2906684541 shutdown_connections 2023-03-03T06:26:19.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.111+0000 7f3824e6d700 1 -- 172.21.15.3:0/2906684541 wait complete. 2023-03-03T06:26:19.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.115+0000 7f3824e6d700 1 Processor -- start 2023-03-03T06:26:19.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.115+0000 7f3824e6d700 1 -- start start 2023-03-03T06:26:19.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f3824e6d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38200d0b30 0x7f38201606c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:19.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f3824e6d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3820156ba0 0x7f38200b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:19.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f3824e6d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f382015d6e0 0x7f38200b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f3824e6d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38200c9610 con 0x7f38200d0b30 2023-03-03T06:26:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f3824e6d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38200cb7a0 con 0x7f3820156ba0 2023-03-03T06:26:19.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f3824e6d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38200caeb0 con 0x7f382015d6e0 2023-03-03T06:26:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f381f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38200d0b30 0x7f38201606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f381f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38200d0b30 0x7f38201606c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45090/0 (socket says 172.21.15.3:45090) 2023-03-03T06:26:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f381f7fe700 1 -- 172.21.15.3:0/3458343376 learned_addr learned my addr 172.21.15.3:0/3458343376 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:19.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f381ffff700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f382015d6e0 0x7f38200b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f381effd700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3820156ba0 0x7f38200b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f381f7fe700 1 -- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f382015d6e0 msgr2=0x7f38200b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f381f7fe700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f382015d6e0 0x7f38200b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f381f7fe700 1 -- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3820156ba0 msgr2=0x7f38200b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f381f7fe700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3820156ba0 0x7f38200b6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f381f7fe700 1 -- 172.21.15.3:0/3458343376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3810009570 con 0x7f38200d0b30 2023-03-03T06:26:19.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.119+0000 7f381f7fe700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38200d0b30 0x7f38201606c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3810000c00 tx=0x7f3810004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.123+0000 7f3807fff700 1 -- 172.21.15.3:0/3458343376 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810011ab0 con 0x7f38200d0b30 2023-03-03T06:26:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.123+0000 7f3807fff700 1 -- 172.21.15.3:0/3458343376 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381001f070 con 0x7f38200d0b30 2023-03-03T06:26:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.123+0000 7f3824e6d700 1 -- 172.21.15.3:0/3458343376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38200717c0 con 0x7f38200d0b30 2023-03-03T06:26:19.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.123+0000 7f3807fff700 1 -- 172.21.15.3:0/3458343376 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810011e50 con 0x7f38200d0b30 2023-03-03T06:26:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.123+0000 7f3824e6d700 1 -- 172.21.15.3:0/3458343376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38200cd8e0 con 0x7f38200d0b30 2023-03-03T06:26:19.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.123+0000 7f3824e6d700 1 -- 172.21.15.3:0/3458343376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38200cff50 con 0x7f38200d0b30 2023-03-03T06:26:19.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.127+0000 7f3807fff700 1 -- 172.21.15.3:0/3458343376 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f381000f050 con 0x7f38200d0b30 2023-03-03T06:26:19.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.127+0000 7f3807fff700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f380c064ee0 0x7f380c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:19.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.127+0000 7f3807fff700 1 -- 172.21.15.3:0/3458343376 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3810021070 con 0x7f38200d0b30 2023-03-03T06:26:19.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.127+0000 7f381effd700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f380c064ee0 0x7f380c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:19.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.131+0000 7f381effd700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f380c064ee0 0x7f380c067390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f38200b2a70 tx=0x7f3814007cb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:19.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.131+0000 7f3807fff700 1 -- 172.21.15.3:0/3458343376 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f3810051b40 con 0x7f38200d0b30 2023-03-03T06:26:19.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.311+0000 7f3824e6d700 1 -- 172.21.15.3:0/3458343376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3820073e90 con 0x7f38200d0b30 2023-03-03T06:26:19.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.311+0000 7f3807fff700 1 -- 172.21.15.3:0/3458343376 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3810015020 con 0x7f38200d0b30 2023-03-03T06:26:19.317 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:19.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.315+0000 7f3824e6d700 1 -- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f380c064ee0 msgr2=0x7f380c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.315+0000 7f3824e6d700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f380c064ee0 0x7f380c067390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f38200b2a70 tx=0x7f3814007cb0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.315+0000 7f3824e6d700 1 -- 172.21.15.3:0/3458343376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38200d0b30 msgr2=0x7f38201606c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.315+0000 7f3824e6d700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38200d0b30 0x7f38201606c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3810000c00 tx=0x7f3810004770 comp rx=0 tx=0).stop 2023-03-03T06:26:19.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.315+0000 7f3824e6d700 1 -- 172.21.15.3:0/3458343376 shutdown_connections 2023-03-03T06:26:19.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.315+0000 7f3824e6d700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f380c064ee0 0x7f380c067390 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.315+0000 7f3824e6d700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f382015d6e0 0x7f38200b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.315+0000 7f3824e6d700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3820156ba0 0x7f38200b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.315+0000 7f3824e6d700 1 --2- 172.21.15.3:0/3458343376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38200d0b30 0x7f38201606c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.315+0000 7f3824e6d700 1 -- 172.21.15.3:0/3458343376 >> 172.21.15.3:0/3458343376 conn(0x7f382006e5c0 msgr2=0x7f382015af20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:19.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.319+0000 7f3824e6d700 1 -- 172.21.15.3:0/3458343376 shutdown_connections 2023-03-03T06:26:19.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.319+0000 7f3824e6d700 1 -- 172.21.15.3:0/3458343376 wait complete. 2023-03-03T06:26:19.357 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-03T06:26:19.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.563+0000 7f9e429df700 1 Processor -- start 2023-03-03T06:26:19.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.563+0000 7f9e429df700 1 -- start start 2023-03-03T06:26:19.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e429df700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e3c160b60 0x7f9e3c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e429df700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c0ccc20 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e429df700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c0cc380 con 0x7f9e3c160790 2023-03-03T06:26:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e429df700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c0cc660 con 0x7f9e3c160b60 2023-03-03T06:26:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e421de700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e3c160b60 0x7f9e3c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:19.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e421de700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9e3c160b60 0x7f9e3c0ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45104/0 (socket says 172.21.15.3:45104) 2023-03-03T06:26:19.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e421de700 1 -- 172.21.15.3:0/1834418576 learned_addr learned my addr 172.21.15.3:0/1834418576 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e421de700 1 -- 172.21.15.3:0/1834418576 >> v1:172.21.15.32:6789/0 conn(0x7f9e3c160790 legacy=0x7f9e3c0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e421de700 1 -- 172.21.15.3:0/1834418576 >> v1:172.21.15.3:6789/0 conn(0x7f9e3c0b0ed0 legacy=0x7f9e3c164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e421de700 1 -- 172.21.15.3:0/1834418576 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0cc940 con 0x7f9e3c160b60 2023-03-03T06:26:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e421de700 1 --2- 172.21.15.3:0/1834418576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e3c160b60 0x7f9e3c0ce060 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009e50 tx=0x7f9e3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97834fd87dd3450a server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e409db700 1 -- 172.21.15.3:0/1834418576 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3400e070 con 0x7f9e3c160b60 2023-03-03T06:26:19.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e409db700 1 -- 172.21.15.3:0/1834418576 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3400b9c0 con 0x7f9e3c160b60 2023-03-03T06:26:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e409db700 1 -- 172.21.15.3:0/1834418576 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3400bd70 con 0x7f9e3c160b60 2023-03-03T06:26:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e429df700 1 -- 172.21.15.3:0/1834418576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e3c160b60 msgr2=0x7f9e3c0ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.567+0000 7f9e429df700 1 --2- 172.21.15.3:0/1834418576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e3c160b60 0x7f9e3c0ce060 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009e50 tx=0x7f9e3400b330 comp rx=0 tx=0).stop 2023-03-03T06:26:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.571+0000 7f9e429df700 1 -- 172.21.15.3:0/1834418576 shutdown_connections 2023-03-03T06:26:19.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.571+0000 7f9e429df700 1 --2- 172.21.15.3:0/1834418576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e3c160b60 0x7f9e3c0ce060 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.571+0000 7f9e429df700 1 -- 172.21.15.3:0/1834418576 >> 172.21.15.3:0/1834418576 conn(0x7f9e3c0607c0 msgr2=0x7f9e3c060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.571+0000 7f9e429df700 1 -- 172.21.15.3:0/1834418576 shutdown_connections 2023-03-03T06:26:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.571+0000 7f9e429df700 1 -- 172.21.15.3:0/1834418576 wait complete. 2023-03-03T06:26:19.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e429df700 1 Processor -- start 2023-03-03T06:26:19.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e429df700 1 -- start start 2023-03-03T06:26:19.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e429df700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e3c0b0ed0 0x7f9e3c1f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:19.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e429df700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3c160790 0x7f9e3c06bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:19.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e429df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c160b60 0x7f9e3c06c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:19.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e429df700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c1f7520 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e429df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0c3750 con 0x7f9e3c160b60 2023-03-03T06:26:19.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e429df700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c162160 con 0x7f9e3c160790 2023-03-03T06:26:19.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e419dd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e3c0b0ed0 0x7f9e3c1f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:19.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e419dd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e3c0b0ed0 0x7f9e3c1f7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45106/0 (socket says 172.21.15.3:45106) 2023-03-03T06:26:19.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e419dd700 1 -- 172.21.15.3:0/4015358041 learned_addr learned my addr 172.21.15.3:0/4015358041 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:19.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e421de700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c160b60 0x7f9e3c06c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:19.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e411dc700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3c160790 0x7f9e3c06bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:19.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e419dd700 1 -- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3c160790 msgr2=0x7f9e3c06bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e419dd700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3c160790 0x7f9e3c06bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e419dd700 1 -- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c160b60 msgr2=0x7f9e3c06c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e419dd700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c160b60 0x7f9e3c06c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e419dd700 1 -- 172.21.15.3:0/4015358041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e34009b30 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.575+0000 7f9e419dd700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e3c0b0ed0 0x7f9e3c1f7710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800c500 tx=0x7f9e3800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:19.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.579+0000 7f9e327fc700 1 -- 172.21.15.3:0/4015358041 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e380052d0 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.579+0000 7f9e327fc700 1 -- 172.21.15.3:0/4015358041 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e38005430 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.579+0000 7f9e429df700 1 -- 172.21.15.3:0/4015358041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c0cb980 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.579+0000 7f9e327fc700 1 -- 172.21.15.3:0/4015358041 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e380055a0 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.579+0000 7f9e429df700 1 -- 172.21.15.3:0/4015358041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c0cb6a0 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.579+0000 7f9e429df700 1 -- 172.21.15.3:0/4015358041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3c0cbea0 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.583+0000 7f9e327fc700 1 -- 172.21.15.3:0/4015358041 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9e38019030 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.587+0000 7f9e327fc700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9e20064f10 0x7f9e200673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.587+0000 7f9e327fc700 1 -- 172.21.15.3:0/4015358041 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9e38081580 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.587+0000 7f9e411dc700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9e20064f10 0x7f9e200673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.587+0000 7f9e411dc700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9e20064f10 0x7f9e200673c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0c5210 tx=0x7f9e2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:19.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.591+0000 7f9e327fc700 1 -- 172.21.15.3:0/4015358041 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9e3804f300 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.767+0000 7f9e429df700 1 -- 172.21.15.3:0/4015358041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e3c073e90 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.767+0000 7f9e327fc700 1 -- 172.21.15.3:0/4015358041 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e38052920 con 0x7f9e3c0b0ed0 2023-03-03T06:26:19.774 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-03-03T06:26:19.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.771+0000 7f9e429df700 1 -- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9e20064f10 msgr2=0x7f9e200673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.771+0000 7f9e429df700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9e20064f10 0x7f9e200673c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0c5210 tx=0x7f9e2c009040 comp rx=0 tx=0).stop 2023-03-03T06:26:19.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.771+0000 7f9e429df700 1 -- 172.21.15.3:0/4015358041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e3c0b0ed0 msgr2=0x7f9e3c1f7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:19.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.771+0000 7f9e429df700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e3c0b0ed0 0x7f9e3c1f7710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800c500 tx=0x7f9e3800c810 comp rx=0 tx=0).stop 2023-03-03T06:26:19.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.771+0000 7f9e429df700 1 -- 172.21.15.3:0/4015358041 shutdown_connections 2023-03-03T06:26:19.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.771+0000 7f9e429df700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9e20064f10 0x7f9e200673c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.771+0000 7f9e429df700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c160b60 0x7f9e3c06c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.771+0000 7f9e429df700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9e3c160790 0x7f9e3c06bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.771+0000 7f9e429df700 1 --2- 172.21.15.3:0/4015358041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e3c0b0ed0 0x7f9e3c1f7710 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:19.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.771+0000 7f9e429df700 1 -- 172.21.15.3:0/4015358041 >> 172.21.15.3:0/4015358041 conn(0x7f9e3c0607c0 msgr2=0x7f9e3c0af770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:19.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.771+0000 7f9e429df700 1 -- 172.21.15.3:0/4015358041 shutdown_connections 2023-03-03T06:26:19.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:19.771+0000 7f9e429df700 1 -- 172.21.15.3:0/4015358041 wait complete. 2023-03-03T06:26:19.811 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-03T06:26:20.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.011+0000 7f90b9c5a700 1 Processor -- start 2023-03-03T06:26:20.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.011+0000 7f90b9c5a700 1 -- start start 2023-03-03T06:26:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90b9c5a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90b4156ba0 0x7f90b4071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90b9c5a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90b415d6e0 0x7f90b40cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:20.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90b9c5a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b40cf3b0 con 0x7f90b40cfd80 2023-03-03T06:26:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90b9c5a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b4156f70 con 0x7f90b4156ba0 2023-03-03T06:26:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90b9c5a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40cd320 con 0x7f90b415d6e0 2023-03-03T06:26:20.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90b9459700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f90b40cfd80 0x7f90b40d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:52500/0 (socket says 172.21.15.3:52500) 2023-03-03T06:26:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90b9459700 1 -- 172.21.15.3:0/2878701180 learned_addr learned my addr 172.21.15.3:0/2878701180 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90b8c58700 1 --2- 172.21.15.3:0/2878701180 >> v2:172.21.15.3:3300/0 conn(0x7f90b4156ba0 0x7f90b4071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90abfff700 1 --2- 172.21.15.3:0/2878701180 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90b415d6e0 0x7f90b40cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90ab7fe700 1 -- 172.21.15.3:0/2878701180 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757180629 0 0) 0x7f90b40cf3b0 con 0x7f90b40cfd80 2023-03-03T06:26:20.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90ab7fe700 1 -- 172.21.15.3:0/2878701180 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f909c0035d0 con 0x7f90b40cfd80 2023-03-03T06:26:20.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90abfff700 1 -- 172.21.15.3:0/2878701180 >> v2:172.21.15.3:3300/0 conn(0x7f90b4156ba0 msgr2=0x7f90b4071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:20.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90abfff700 1 --2- 172.21.15.3:0/2878701180 >> v2:172.21.15.3:3300/0 conn(0x7f90b4156ba0 0x7f90b4071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90abfff700 1 -- 172.21.15.3:0/2878701180 >> v1:172.21.15.3:6789/0 conn(0x7f90b40cfd80 legacy=0x7f90b40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:20.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90abfff700 1 -- 172.21.15.3:0/2878701180 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b40cd600 con 0x7f90b415d6e0 2023-03-03T06:26:20.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90b8c58700 1 --2- 172.21.15.3:0/2878701180 >> v2:172.21.15.3:3300/0 conn(0x7f90b4156ba0 0x7f90b4071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:20.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.015+0000 7f90abfff700 1 --2- 172.21.15.3:0/2878701180 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90b415d6e0 0x7f90b40cf840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009850 tx=0x7f90a4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b821b0b67730148 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.019+0000 7f90ab7fe700 1 -- 172.21.15.3:0/2878701180 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4011830 con 0x7f90b415d6e0 2023-03-03T06:26:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.019+0000 7f90ab7fe700 1 -- 172.21.15.3:0/2878701180 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a4011990 con 0x7f90b415d6e0 2023-03-03T06:26:20.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.019+0000 7f90ab7fe700 1 -- 172.21.15.3:0/2878701180 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4011d40 con 0x7f90b415d6e0 2023-03-03T06:26:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.019+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2878701180 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90b415d6e0 msgr2=0x7f90b40cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.019+0000 7f90b9c5a700 1 --2- 172.21.15.3:0/2878701180 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90b415d6e0 0x7f90b40cf840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009850 tx=0x7f90a4009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.019+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2878701180 shutdown_connections 2023-03-03T06:26:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.019+0000 7f90b9c5a700 1 --2- 172.21.15.3:0/2878701180 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90b415d6e0 0x7f90b40cf840 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.019+0000 7f90b9c5a700 1 --2- 172.21.15.3:0/2878701180 >> v2:172.21.15.3:3300/0 conn(0x7f90b4156ba0 0x7f90b4071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.019+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2878701180 >> 172.21.15.3:0/2878701180 conn(0x7f90b406e5c0 msgr2=0x7f90b406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:20.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.019+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2878701180 shutdown_connections 2023-03-03T06:26:20.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.019+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2878701180 wait complete. 2023-03-03T06:26:20.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.023+0000 7f90b9c5a700 1 Processor -- start 2023-03-03T06:26:20.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.023+0000 7f90b9c5a700 1 -- start start 2023-03-03T06:26:20.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.023+0000 7f90b9c5a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90b40cfd80 0x7f90b40cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:20.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.023+0000 7f90b9c5a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90b4156ba0 0x7f90b4157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:20.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.023+0000 7f90b9c5a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90b415d6e0 0x7f90b40c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:20.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.023+0000 7f90b9c5a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b414c930 con 0x7f90b4156ba0 2023-03-03T06:26:20.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.023+0000 7f90b9c5a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40b6ab0 con 0x7f90b415d6e0 2023-03-03T06:26:20.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.023+0000 7f90b9c5a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40b7630 con 0x7f90b40cfd80 2023-03-03T06:26:20.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.023+0000 7f90abfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90b4156ba0 0x7f90b4157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:20.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.023+0000 7f90abfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90b4156ba0 0x7f90b4157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45132/0 (socket says 172.21.15.3:45132) 2023-03-03T06:26:20.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.023+0000 7f90abfff700 1 -- 172.21.15.3:0/2736712407 learned_addr learned my addr 172.21.15.3:0/2736712407 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:20.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.023+0000 7f90b8c58700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90b40cfd80 0x7f90b40cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:20.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.027+0000 7f90b9459700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90b415d6e0 0x7f90b40c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:20.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.027+0000 7f90abfff700 1 -- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90b40cfd80 msgr2=0x7f90b40cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:20.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.027+0000 7f90abfff700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90b40cfd80 0x7f90b40cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.027+0000 7f90abfff700 1 -- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90b415d6e0 msgr2=0x7f90b40c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:20.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.027+0000 7f90abfff700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90b415d6e0 0x7f90b40c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.027+0000 7f90abfff700 1 -- 172.21.15.3:0/2736712407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b00094b0 con 0x7f90b4156ba0 2023-03-03T06:26:20.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.027+0000 7f90abfff700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90b4156ba0 0x7f90b4157df0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f90b415a660 tx=0x7f90a4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:20.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.027+0000 7f90a97fa700 1 -- 172.21.15.3:0/2736712407 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4004380 con 0x7f90b4156ba0 2023-03-03T06:26:20.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.027+0000 7f90a97fa700 1 -- 172.21.15.3:0/2736712407 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a401f070 con 0x7f90b4156ba0 2023-03-03T06:26:20.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.027+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2736712407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a40094b0 con 0x7f90b4156ba0 2023-03-03T06:26:20.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.027+0000 7f90a97fa700 1 -- 172.21.15.3:0/2736712407 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a40036c0 con 0x7f90b4156ba0 2023-03-03T06:26:20.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.027+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2736712407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b40717c0 con 0x7f90b4156ba0 2023-03-03T06:26:20.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.031+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2736712407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b40cd8e0 con 0x7f90b4156ba0 2023-03-03T06:26:20.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.031+0000 7f90a97fa700 1 -- 172.21.15.3:0/2736712407 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f90a402a020 con 0x7f90b4156ba0 2023-03-03T06:26:20.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.035+0000 7f90a97fa700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f90a0064f10 0x7f90a00673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:20.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.035+0000 7f90a97fa700 1 -- 172.21.15.3:0/2736712407 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f90a4083720 con 0x7f90b4156ba0 2023-03-03T06:26:20.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.035+0000 7f90b8c58700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f90a0064f10 0x7f90a00673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:20.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.035+0000 7f90b8c58700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f90a0064f10 0x7f90a00673c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009d80 tx=0x7f90b0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:20.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.039+0000 7f90a97fa700 1 -- 172.21.15.3:0/2736712407 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f90a40514a0 con 0x7f90b4156ba0 2023-03-03T06:26:20.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.215+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2736712407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f90b40cef40 con 0x7f90b4156ba0 2023-03-03T06:26:20.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.219+0000 7f90a97fa700 1 -- 172.21.15.3:0/2736712407 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f90a4006ce0 con 0x7f90b4156ba0 2023-03-03T06:26:20.222 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:20.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.219+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f90a0064f10 msgr2=0x7f90a00673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:20.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.223+0000 7f90b9c5a700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f90a0064f10 0x7f90a00673c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009d80 tx=0x7f90b0009240 comp rx=0 tx=0).stop 2023-03-03T06:26:20.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.223+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2736712407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90b4156ba0 msgr2=0x7f90b4157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:20.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.223+0000 7f90b9c5a700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90b4156ba0 0x7f90b4157df0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f90b415a660 tx=0x7f90a4004770 comp rx=0 tx=0).stop 2023-03-03T06:26:20.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.223+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2736712407 shutdown_connections 2023-03-03T06:26:20.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.223+0000 7f90b9c5a700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f90a0064f10 0x7f90a00673c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.223+0000 7f90b9c5a700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f90b415d6e0 0x7f90b40c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.223+0000 7f90b9c5a700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90b4156ba0 0x7f90b4157df0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.223+0000 7f90b9c5a700 1 --2- 172.21.15.3:0/2736712407 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90b40cfd80 0x7f90b40cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.223+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2736712407 >> 172.21.15.3:0/2736712407 conn(0x7f90b406e5c0 msgr2=0x7f90b415ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:20.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.223+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2736712407 shutdown_connections 2023-03-03T06:26:20.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.223+0000 7f90b9c5a700 1 -- 172.21.15.3:0/2736712407 wait complete. 2023-03-03T06:26:20.261 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-03T06:26:20.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.467+0000 7fa72d5fc700 1 Processor -- start 2023-03-03T06:26:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.467+0000 7fa72d5fc700 1 -- start start 2023-03-03T06:26:20.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.471+0000 7fa72d5fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa72815fc60 0x7fa72815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:20.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.471+0000 7fa72d5fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7280d3a90 0x7fa7280d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:20.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.471+0000 7fa72d5fc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7280cf970 con 0x7fa7280d21c0 2023-03-03T06:26:20.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.471+0000 7fa72d5fc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72815ce40 con 0x7fa72815fc60 2023-03-03T06:26:20.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.471+0000 7fa72d5fc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72815d120 con 0x7fa7280d3a90 2023-03-03T06:26:20.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.471+0000 7fa7267fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa7280d21c0 0x7fa7280d0df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53632/0 (socket says 172.21.15.3:53632) 2023-03-03T06:26:20.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.471+0000 7fa7267fc700 1 -- 172.21.15.3:0/570411788 learned_addr learned my addr 172.21.15.3:0/570411788 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:20.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.471+0000 7fa727fff700 1 --2- 172.21.15.3:0/570411788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7280d3a90 0x7fa7280d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:20.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.471+0000 7fa726ffd700 1 --2- 172.21.15.3:0/570411788 >> v2:172.21.15.3:3300/0 conn(0x7fa72815fc60 0x7fa72815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:20.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.471+0000 7fa725ffb700 1 -- 172.21.15.3:0/570411788 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360044744 0 0) 0x7fa7280cf970 con 0x7fa7280d21c0 2023-03-03T06:26:20.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.475+0000 7fa725ffb700 1 -- 172.21.15.3:0/570411788 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7100035d0 con 0x7fa7280d21c0 2023-03-03T06:26:20.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.475+0000 7fa727fff700 1 -- 172.21.15.3:0/570411788 >> v2:172.21.15.3:3300/0 conn(0x7fa72815fc60 msgr2=0x7fa72815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:20.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.475+0000 7fa727fff700 1 --2- 172.21.15.3:0/570411788 >> v2:172.21.15.3:3300/0 conn(0x7fa72815fc60 0x7fa72815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.475+0000 7fa727fff700 1 -- 172.21.15.3:0/570411788 >> v1:172.21.15.3:6789/0 conn(0x7fa7280d21c0 legacy=0x7fa7280d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:20.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.475+0000 7fa727fff700 1 -- 172.21.15.3:0/570411788 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72815d400 con 0x7fa7280d3a90 2023-03-03T06:26:20.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.475+0000 7fa727fff700 1 --2- 172.21.15.3:0/570411788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7280d3a90 0x7fa7280d29f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa720009d90 tx=0x7fa72000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b7b984370dd776 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:20.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.475+0000 7fa725ffb700 1 -- 172.21.15.3:0/570411788 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72000e070 con 0x7fa7280d3a90 2023-03-03T06:26:20.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.475+0000 7fa725ffb700 1 -- 172.21.15.3:0/570411788 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72000bb20 con 0x7fa7280d3a90 2023-03-03T06:26:20.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.475+0000 7fa725ffb700 1 -- 172.21.15.3:0/570411788 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720012430 con 0x7fa7280d3a90 2023-03-03T06:26:20.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.475+0000 7fa72d5fc700 1 -- 172.21.15.3:0/570411788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7280d3a90 msgr2=0x7fa7280d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:20.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.475+0000 7fa72d5fc700 1 --2- 172.21.15.3:0/570411788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7280d3a90 0x7fa7280d29f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa720009d90 tx=0x7fa72000b330 comp rx=0 tx=0).stop 2023-03-03T06:26:20.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.475+0000 7fa72d5fc700 1 -- 172.21.15.3:0/570411788 shutdown_connections 2023-03-03T06:26:20.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.475+0000 7fa72d5fc700 1 --2- 172.21.15.3:0/570411788 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7280d3a90 0x7fa7280d29f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.479+0000 7fa72d5fc700 1 --2- 172.21.15.3:0/570411788 >> v2:172.21.15.3:3300/0 conn(0x7fa72815fc60 0x7fa72815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.479+0000 7fa72d5fc700 1 -- 172.21.15.3:0/570411788 >> 172.21.15.3:0/570411788 conn(0x7fa72806b580 msgr2=0x7fa72806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:20.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.479+0000 7fa72d5fc700 1 -- 172.21.15.3:0/570411788 shutdown_connections 2023-03-03T06:26:20.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.479+0000 7fa72d5fc700 1 -- 172.21.15.3:0/570411788 wait complete. 2023-03-03T06:26:20.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa72d5fc700 1 Processor -- start 2023-03-03T06:26:20.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa72d5fc700 1 -- start start 2023-03-03T06:26:20.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa72d5fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7280d21c0 0x7fa7280b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:20.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa72d5fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7280d3a90 0x7fa72815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:20.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa72d5fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72815fc60 0x7fa72815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:20.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa72d5fc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72806f350 con 0x7fa7280d3a90 2023-03-03T06:26:20.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa72d5fc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7280b6ee0 con 0x7fa72815fc60 2023-03-03T06:26:20.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa72d5fc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72806e6d0 con 0x7fa7280d21c0 2023-03-03T06:26:20.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa7267fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7280d3a90 0x7fa72815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:20.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa726ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7280d21c0 0x7fa7280b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:20.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa7267fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7280d3a90 0x7fa72815e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44604/0 (socket says 172.21.15.3:44604) 2023-03-03T06:26:20.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa7267fc700 1 -- 172.21.15.3:0/214704710 learned_addr learned my addr 172.21.15.3:0/214704710 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:20.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa727fff700 1 --2- 172.21.15.3:0/214704710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72815fc60 0x7fa72815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:20.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa7267fc700 1 -- 172.21.15.3:0/214704710 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7280d21c0 msgr2=0x7fa7280b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:20.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa7267fc700 1 --2- 172.21.15.3:0/214704710 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7280d21c0 0x7fa7280b69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa7267fc700 1 -- 172.21.15.3:0/214704710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72815fc60 msgr2=0x7fa72815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:20.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa7267fc700 1 --2- 172.21.15.3:0/214704710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72815fc60 0x7fa72815e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.483+0000 7fa7267fc700 1 -- 172.21.15.3:0/214704710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa720009a70 con 0x7fa7280d3a90 2023-03-03T06:26:20.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.487+0000 7fa7267fc700 1 --2- 172.21.15.3:0/214704710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7280d3a90 0x7fa72815e4b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00c4d0 tx=0x7fa71c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:20.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.487+0000 7fa70f7fe700 1 -- 172.21.15.3:0/214704710 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c005d90 con 0x7fa7280d3a90 2023-03-03T06:26:20.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.487+0000 7fa70f7fe700 1 -- 172.21.15.3:0/214704710 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c00ce80 con 0x7fa7280d3a90 2023-03-03T06:26:20.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.487+0000 7fa72d5fc700 1 -- 172.21.15.3:0/214704710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72815da50 con 0x7fa7280d3a90 2023-03-03T06:26:20.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.487+0000 7fa70f7fe700 1 -- 172.21.15.3:0/214704710 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c007310 con 0x7fa7280d3a90 2023-03-03T06:26:20.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.487+0000 7fa72d5fc700 1 -- 172.21.15.3:0/214704710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72815d6e0 con 0x7fa7280d3a90 2023-03-03T06:26:20.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.491+0000 7fa72d5fc700 1 -- 172.21.15.3:0/214704710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72815caa0 con 0x7fa7280d3a90 2023-03-03T06:26:20.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.491+0000 7fa70f7fe700 1 -- 172.21.15.3:0/214704710 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa71c019030 con 0x7fa7280d3a90 2023-03-03T06:26:20.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.491+0000 7fa70f7fe700 1 --2- 172.21.15.3:0/214704710 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa714064ee0 0x7fa714067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:20.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.491+0000 7fa70f7fe700 1 -- 172.21.15.3:0/214704710 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa71c0814f0 con 0x7fa7280d3a90 2023-03-03T06:26:20.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.495+0000 7fa726ffd700 1 --2- 172.21.15.3:0/214704710 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa714064ee0 0x7fa714067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:20.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.495+0000 7fa726ffd700 1 --2- 172.21.15.3:0/214704710 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa714064ee0 0x7fa714067390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa728158830 tx=0x7fa718009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:20.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.499+0000 7fa70f7fe700 1 -- 172.21.15.3:0/214704710 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa71c04f270 con 0x7fa7280d3a90 2023-03-03T06:26:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.687+0000 7fa72d5fc700 1 -- 172.21.15.3:0/214704710 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa72804e180 con 0x7fa7280d3a90 2023-03-03T06:26:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.691+0000 7fa70f7fe700 1 -- 172.21.15.3:0/214704710 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa71c052890 con 0x7fa7280d3a90 2023-03-03T06:26:20.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.695+0000 7fa72d5fc700 1 -- 172.21.15.3:0/214704710 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa714064ee0 msgr2=0x7fa714067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:20.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.695+0000 7fa72d5fc700 1 --2- 172.21.15.3:0/214704710 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa714064ee0 0x7fa714067390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa728158830 tx=0x7fa718009240 comp rx=0 tx=0).stop 2023-03-03T06:26:20.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.695+0000 7fa72d5fc700 1 -- 172.21.15.3:0/214704710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7280d3a90 msgr2=0x7fa72815e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:20.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.695+0000 7fa72d5fc700 1 --2- 172.21.15.3:0/214704710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7280d3a90 0x7fa72815e4b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00c4d0 tx=0x7fa71c00c7e0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.695+0000 7fa72d5fc700 1 -- 172.21.15.3:0/214704710 shutdown_connections 2023-03-03T06:26:20.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.695+0000 7fa72d5fc700 1 --2- 172.21.15.3:0/214704710 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa714064ee0 0x7fa714067390 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.695+0000 7fa72d5fc700 1 --2- 172.21.15.3:0/214704710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa72815fc60 0x7fa72815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.695+0000 7fa72d5fc700 1 --2- 172.21.15.3:0/214704710 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7280d3a90 0x7fa72815e4b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.695+0000 7fa72d5fc700 1 --2- 172.21.15.3:0/214704710 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa7280d21c0 0x7fa7280b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:20.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.695+0000 7fa72d5fc700 1 -- 172.21.15.3:0/214704710 >> 172.21.15.3:0/214704710 conn(0x7fa72806b580 msgr2=0x7fa728159790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:20.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.695+0000 7fa72d5fc700 1 -- 172.21.15.3:0/214704710 shutdown_connections 2023-03-03T06:26:20.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:20.695+0000 7fa72d5fc700 1 -- 172.21.15.3:0/214704710 wait complete. 2023-03-03T06:26:20.703 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2023-03-03T06:26:20.730 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-03-03T06:26:20.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:20.731 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-03-03T06:26:20.820 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-03-03T06:26:20.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:20.822 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-03-03T06:26:20.846 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:26:20.846 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:20.869 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:26:20.870 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:20.936 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:26:20.936 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:21.005 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:21.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.211+0000 7fb3a14fa700 1 Processor -- start 2023-03-03T06:26:21.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.211+0000 7fb3a14fa700 1 -- start start 2023-03-03T06:26:21.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.211+0000 7fb3a14fa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb39c0aee70 0x7fb39c158440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:21.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.211+0000 7fb3a14fa700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb39c15c040 0x7fb39c15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:21.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.211+0000 7fb3a14fa700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39c0cdd30 con 0x7fb39c06a780 2023-03-03T06:26:21.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.211+0000 7fb3a14fa700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39c15ca20 con 0x7fb39c15c040 2023-03-03T06:26:21.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.211+0000 7fb3a14fa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c158980 con 0x7fb39c0aee70 2023-03-03T06:26:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb3a0cf9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb39c15c040 0x7fb39c15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb3937fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb39c0aee70 0x7fb39c158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb3937fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb39c0aee70 0x7fb39c158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:43064/0 (socket says 172.21.15.3:43064) 2023-03-03T06:26:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb3a0cf9700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb39c15c040 0x7fb39c15c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44606/0 (socket says 172.21.15.3:44606) 2023-03-03T06:26:21.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb3937fe700 1 -- 172.21.15.3:0/2936241533 learned_addr learned my addr 172.21.15.3:0/2936241533 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb3a0cf9700 1 -- 172.21.15.3:0/2936241533 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb39c0aee70 msgr2=0x7fb39c158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb3a0cf9700 1 --2- 172.21.15.3:0/2936241533 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb39c0aee70 0x7fb39c158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb3a0cf9700 1 -- 172.21.15.3:0/2936241533 >> v1:172.21.15.32:6789/0 conn(0x7fb39c06a780 legacy=0x7fb39c0c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb3a0cf9700 1 -- 172.21.15.3:0/2936241533 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c0ca4d0 con 0x7fb39c15c040 2023-03-03T06:26:21.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb3937fe700 1 --2- 172.21.15.3:0/2936241533 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb39c0aee70 0x7fb39c158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb3a0cf9700 1 --2- 172.21.15.3:0/2936241533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb39c15c040 0x7fb39c15c4e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb394009850 tx=0x7fb394009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d77529513fe8ce server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:21.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb39b7fe700 1 -- 172.21.15.3:0/2936241533 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394011830 con 0x7fb39c15c040 2023-03-03T06:26:21.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb39b7fe700 1 -- 172.21.15.3:0/2936241533 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb394011990 con 0x7fb39c15c040 2023-03-03T06:26:21.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb39b7fe700 1 -- 172.21.15.3:0/2936241533 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394011d40 con 0x7fb39c15c040 2023-03-03T06:26:21.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2936241533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb39c15c040 msgr2=0x7fb39c15c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:21.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.215+0000 7fb3a14fa700 1 --2- 172.21.15.3:0/2936241533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb39c15c040 0x7fb39c15c4e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb394009850 tx=0x7fb394009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:21.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.219+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2936241533 shutdown_connections 2023-03-03T06:26:21.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.219+0000 7fb3a14fa700 1 --2- 172.21.15.3:0/2936241533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb39c15c040 0x7fb39c15c4e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.219+0000 7fb3a14fa700 1 --2- 172.21.15.3:0/2936241533 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb39c0aee70 0x7fb39c158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.219+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2936241533 >> 172.21.15.3:0/2936241533 conn(0x7fb39c070a30 msgr2=0x7fb39c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:21.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.219+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2936241533 shutdown_connections 2023-03-03T06:26:21.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.219+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2936241533 wait complete. 2023-03-03T06:26:21.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.219+0000 7fb3a14fa700 1 Processor -- start 2023-03-03T06:26:21.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a14fa700 1 -- start start 2023-03-03T06:26:21.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a14fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb39c06a780 0x7fb39c146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:21.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a14fa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb39c0aee70 0x7fb39c0b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:21.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a14fa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb39c15c040 0x7fb39c147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:21.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a14fa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c147400 con 0x7fb39c15c040 2023-03-03T06:26:21.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a14fa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c14b0a0 con 0x7fb39c06a780 2023-03-03T06:26:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a14fa700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c14a800 con 0x7fb39c0aee70 2023-03-03T06:26:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a0cf9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb39c15c040 0x7fb39c147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a0cf9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb39c15c040 0x7fb39c147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44620/0 (socket says 172.21.15.3:44620) 2023-03-03T06:26:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a0cf9700 1 -- 172.21.15.3:0/2734272276 learned_addr learned my addr 172.21.15.3:0/2734272276 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3937fe700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb39c0aee70 0x7fb39c0b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb39bfff700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb39c06a780 0x7fb39c146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:21.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a0cf9700 1 -- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb39c0aee70 msgr2=0x7fb39c0b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a0cf9700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb39c0aee70 0x7fb39c0b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a0cf9700 1 -- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb39c06a780 msgr2=0x7fb39c146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a0cf9700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb39c06a780 0x7fb39c146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.223+0000 7fb3a0cf9700 1 -- 172.21.15.3:0/2734272276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb384009570 con 0x7fb39c15c040 2023-03-03T06:26:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.227+0000 7fb3a0cf9700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb39c15c040 0x7fb39c147680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb3940090b0 tx=0x7fb394004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.227+0000 7fb3997fa700 1 -- 172.21.15.3:0/2734272276 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940036c0 con 0x7fb39c15c040 2023-03-03T06:26:21.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.227+0000 7fb3997fa700 1 -- 172.21.15.3:0/2734272276 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39401e070 con 0x7fb39c15c040 2023-03-03T06:26:21.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.227+0000 7fb3997fa700 1 -- 172.21.15.3:0/2734272276 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394003a40 con 0x7fb39c15c040 2023-03-03T06:26:21.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.227+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2734272276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3940094b0 con 0x7fb39c15c040 2023-03-03T06:26:21.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.227+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2734272276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39c0cd9c0 con 0x7fb39c15c040 2023-03-03T06:26:21.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.227+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2734272276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39c0cd6e0 con 0x7fb39c15c040 2023-03-03T06:26:21.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.231+0000 7fb3997fa700 1 -- 172.21.15.3:0/2734272276 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb39400f050 con 0x7fb39c15c040 2023-03-03T06:26:21.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.231+0000 7fb3997fa700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb37c064f10 0x7fb37c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:21.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.231+0000 7fb3997fa700 1 -- 172.21.15.3:0/2734272276 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb394020070 con 0x7fb39c15c040 2023-03-03T06:26:21.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.231+0000 7fb39bfff700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb37c064f10 0x7fb37c0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:21.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.235+0000 7fb39bfff700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb37c064f10 0x7fb37c0673c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0012e0 tx=0x7fb38c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:21.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.235+0000 7fb3997fa700 1 -- 172.21.15.3:0/2734272276 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb394051550 con 0x7fb39c15c040 2023-03-03T06:26:21.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.419+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2734272276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb39c0ca7b0 con 0x7fb39c15c040 2023-03-03T06:26:21.426 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.423+0000 7fb3997fa700 1 -- 172.21.15.3:0/2734272276 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19652 (secure 0 0 0) 0x7fb394015090 con 0x7fb39c15c040 2023-03-03T06:26:21.426 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:21.427 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:53.877945+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.427+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb37c064f10 msgr2=0x7fb37c0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.427+0000 7fb3a14fa700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb37c064f10 0x7fb37c0673c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0012e0 tx=0x7fb38c009040 comp rx=0 tx=0).stop 2023-03-03T06:26:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.427+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2734272276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb39c15c040 msgr2=0x7fb39c147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:21.432 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.427+0000 7fb3a14fa700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb39c15c040 0x7fb39c147680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb3940090b0 tx=0x7fb394004770 comp rx=0 tx=0).stop 2023-03-03T06:26:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.427+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2734272276 shutdown_connections 2023-03-03T06:26:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.427+0000 7fb3a14fa700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb37c064f10 0x7fb37c0673c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.427+0000 7fb3a14fa700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb39c15c040 0x7fb39c147680 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.427+0000 7fb3a14fa700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb39c0aee70 0x7fb39c0b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.427+0000 7fb3a14fa700 1 --2- 172.21.15.3:0/2734272276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb39c06a780 0x7fb39c146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.427+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2734272276 >> 172.21.15.3:0/2734272276 conn(0x7fb39c070a30 msgr2=0x7fb39c06bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.427+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2734272276 shutdown_connections 2023-03-03T06:26:21.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.427+0000 7fb3a14fa700 1 -- 172.21.15.3:0/2734272276 wait complete. 2023-03-03T06:26:21.467 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-03T06:25:11.468534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-03T06:25:12.813283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-03T06:25:16.382800+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-03T06:26:21.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:26:21.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.675+0000 7f7de9c34700 1 Processor -- start 2023-03-03T06:26:21.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.679+0000 7f7de9c34700 1 -- start start 2023-03-03T06:26:21.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.679+0000 7f7de9c34700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7de4156ba0 0x7f7de4071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:21.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.679+0000 7f7de9c34700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7de415d6e0 0x7f7de40cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:21.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.679+0000 7f7de9c34700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7de40cfd80 0x7f7de40d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:21.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.679+0000 7f7de9c34700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de40cf3b0 con 0x7f7de415d6e0 2023-03-03T06:26:21.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.679+0000 7f7de9c34700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de4156f70 con 0x7f7de40cfd80 2023-03-03T06:26:21.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.679+0000 7f7de9c34700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de40cd320 con 0x7f7de4156ba0 2023-03-03T06:26:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7de415d6e0 0x7f7de40cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7de415d6e0 0x7f7de40cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44628/0 (socket says 172.21.15.3:44628) 2023-03-03T06:26:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de3fff700 1 -- 172.21.15.3:0/2496987376 learned_addr learned my addr 172.21.15.3:0/2496987376 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de8c32700 1 --2- 172.21.15.3:0/2496987376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7de4156ba0 0x7f7de4071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de9433700 1 --2- 172.21.15.3:0/2496987376 >> v2:172.21.15.32:3300/0 conn(0x7f7de40cfd80 0x7f7de40d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de3fff700 1 -- 172.21.15.3:0/2496987376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7de4156ba0 msgr2=0x7f7de4071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de3fff700 1 --2- 172.21.15.3:0/2496987376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7de4156ba0 0x7f7de4071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de3fff700 1 -- 172.21.15.3:0/2496987376 >> v2:172.21.15.32:3300/0 conn(0x7f7de40cfd80 msgr2=0x7f7de40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de3fff700 1 --2- 172.21.15.3:0/2496987376 >> v2:172.21.15.32:3300/0 conn(0x7f7de40cfd80 0x7f7de40d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de3fff700 1 -- 172.21.15.3:0/2496987376 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de40cd600 con 0x7f7de415d6e0 2023-03-03T06:26:21.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de3fff700 1 --2- 172.21.15.3:0/2496987376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de415d6e0 0x7f7de40cf840 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7de40c9c90 tx=0x7f7dd0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c2d7b2a6b4556a server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:21.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de37fe700 1 -- 172.21.15.3:0/2496987376 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0011830 con 0x7f7de415d6e0 2023-03-03T06:26:21.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de37fe700 1 -- 172.21.15.3:0/2496987376 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd0011990 con 0x7f7de415d6e0 2023-03-03T06:26:21.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de37fe700 1 -- 172.21.15.3:0/2496987376 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0011d20 con 0x7f7de415d6e0 2023-03-03T06:26:21.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de9c34700 1 -- 172.21.15.3:0/2496987376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de415d6e0 msgr2=0x7f7de40cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:21.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.683+0000 7f7de9c34700 1 --2- 172.21.15.3:0/2496987376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de415d6e0 0x7f7de40cf840 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7de40c9c90 tx=0x7f7dd0009bf0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.687+0000 7f7de9c34700 1 -- 172.21.15.3:0/2496987376 shutdown_connections 2023-03-03T06:26:21.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.687+0000 7f7de9c34700 1 --2- 172.21.15.3:0/2496987376 >> v2:172.21.15.32:3300/0 conn(0x7f7de40cfd80 0x7f7de40d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.687+0000 7f7de9c34700 1 --2- 172.21.15.3:0/2496987376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de415d6e0 0x7f7de40cf840 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.687+0000 7f7de9c34700 1 --2- 172.21.15.3:0/2496987376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7de4156ba0 0x7f7de4071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.687+0000 7f7de9c34700 1 -- 172.21.15.3:0/2496987376 >> 172.21.15.3:0/2496987376 conn(0x7f7de406e5c0 msgr2=0x7f7de406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:21.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.687+0000 7f7de9c34700 1 -- 172.21.15.3:0/2496987376 shutdown_connections 2023-03-03T06:26:21.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.687+0000 7f7de9c34700 1 -- 172.21.15.3:0/2496987376 wait complete. 2023-03-03T06:26:21.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.687+0000 7f7de9c34700 1 Processor -- start 2023-03-03T06:26:21.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.687+0000 7f7de9c34700 1 -- start start 2023-03-03T06:26:21.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de9c34700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7de40cfd80 0x7f7de41551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de9c34700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de4156ba0 0x7f7de4155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de9c34700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7de415d6e0 0x7f7de41f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de9c34700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de40c6760 con 0x7f7de4156ba0 2023-03-03T06:26:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de9c34700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de40c7580 con 0x7f7de415d6e0 2023-03-03T06:26:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de9c34700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de4155dc0 con 0x7f7de40cfd80 2023-03-03T06:26:21.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de4156ba0 0x7f7de4155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:21.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de4156ba0 0x7f7de4155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44636/0 (socket says 172.21.15.3:44636) 2023-03-03T06:26:21.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de3fff700 1 -- 172.21.15.3:0/4226200637 learned_addr learned my addr 172.21.15.3:0/4226200637 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:21.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de8c32700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7de40cfd80 0x7f7de41551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:21.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de9433700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7de415d6e0 0x7f7de41f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:21.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de3fff700 1 -- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7de40cfd80 msgr2=0x7f7de41551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:21.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de3fff700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7de40cfd80 0x7f7de41551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de3fff700 1 -- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7de415d6e0 msgr2=0x7f7de41f1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:21.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de3fff700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7de415d6e0 0x7f7de41f1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.691+0000 7f7de3fff700 1 -- 172.21.15.3:0/4226200637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd0009570 con 0x7f7de4156ba0 2023-03-03T06:26:21.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.695+0000 7f7de3fff700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de4156ba0 0x7f7de4155880 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7de406f190 tx=0x7f7dd0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:21.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.695+0000 7f7de17fa700 1 -- 172.21.15.3:0/4226200637 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0004240 con 0x7f7de4156ba0 2023-03-03T06:26:21.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.695+0000 7f7de17fa700 1 -- 172.21.15.3:0/4226200637 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd001f070 con 0x7f7de4156ba0 2023-03-03T06:26:21.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.695+0000 7f7de17fa700 1 -- 172.21.15.3:0/4226200637 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0004600 con 0x7f7de4156ba0 2023-03-03T06:26:21.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.695+0000 7f7de9c34700 1 -- 172.21.15.3:0/4226200637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de40717c0 con 0x7f7de4156ba0 2023-03-03T06:26:21.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.695+0000 7f7de9c34700 1 -- 172.21.15.3:0/4226200637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de40cd8e0 con 0x7f7de4156ba0 2023-03-03T06:26:21.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.695+0000 7f7de9c34700 1 -- 172.21.15.3:0/4226200637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de40cef40 con 0x7f7de4156ba0 2023-03-03T06:26:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.699+0000 7f7de17fa700 1 -- 172.21.15.3:0/4226200637 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7dd000f050 con 0x7f7de4156ba0 2023-03-03T06:26:21.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.699+0000 7f7de17fa700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7dc8064ee0 0x7f7dc8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:21.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.699+0000 7f7de17fa700 1 -- 172.21.15.3:0/4226200637 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7dd00833c0 con 0x7f7de4156ba0 2023-03-03T06:26:21.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.703+0000 7f7de8c32700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7dc8064ee0 0x7f7dc8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:21.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.703+0000 7f7de8c32700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7dc8064ee0 0x7f7dc8067390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009a00 tx=0x7f7dd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:21.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.707+0000 7f7de17fa700 1 -- 172.21.15.3:0/4226200637 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7dd0051200 con 0x7f7de4156ba0 2023-03-03T06:26:21.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.887+0000 7f7de9c34700 1 -- 172.21.15.3:0/4226200637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7de4073e90 con 0x7f7de4156ba0 2023-03-03T06:26:21.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.887+0000 7f7de17fa700 1 -- 172.21.15.3:0/4226200637 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7dd0015020 con 0x7f7de4156ba0 2023-03-03T06:26:21.898 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:26:21.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.891+0000 7f7de9c34700 1 -- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7dc8064ee0 msgr2=0x7f7dc8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:21.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.891+0000 7f7de9c34700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7dc8064ee0 0x7f7dc8067390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009a00 tx=0x7f7dd8008040 comp rx=0 tx=0).stop 2023-03-03T06:26:21.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.891+0000 7f7de9c34700 1 -- 172.21.15.3:0/4226200637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de4156ba0 msgr2=0x7f7de4155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:21.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.891+0000 7f7de9c34700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de4156ba0 0x7f7de4155880 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7de406f190 tx=0x7f7dd0004770 comp rx=0 tx=0).stop 2023-03-03T06:26:21.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.895+0000 7f7de9c34700 1 -- 172.21.15.3:0/4226200637 shutdown_connections 2023-03-03T06:26:21.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.895+0000 7f7de9c34700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7dc8064ee0 0x7f7dc8067390 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.895+0000 7f7de9c34700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7de415d6e0 0x7f7de41f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.895+0000 7f7de9c34700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de4156ba0 0x7f7de4155880 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.895+0000 7f7de9c34700 1 --2- 172.21.15.3:0/4226200637 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7de40cfd80 0x7f7de41551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:21.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.895+0000 7f7de9c34700 1 -- 172.21.15.3:0/4226200637 >> 172.21.15.3:0/4226200637 conn(0x7f7de406e5c0 msgr2=0x7f7de415e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:21.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.895+0000 7f7de9c34700 1 -- 172.21.15.3:0/4226200637 shutdown_connections 2023-03-03T06:26:21.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:21.895+0000 7f7de9c34700 1 -- 172.21.15.3:0/4226200637 wait complete. 2023-03-03T06:26:21.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:26:22.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.139+0000 7f91bee0f700 1 Processor -- start 2023-03-03T06:26:22.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.139+0000 7f91bee0f700 1 -- start start 2023-03-03T06:26:22.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.143+0000 7f91bee0f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91b80c95c0 0x7f91b806d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:22.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.143+0000 7f91bee0f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b80ccf20 con 0x7f91b80c9a10 2023-03-03T06:26:22.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.143+0000 7f91bee0f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b80cd200 con 0x7f91b815fb60 2023-03-03T06:26:22.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.143+0000 7f91bee0f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b815ff30 con 0x7f91b80c95c0 2023-03-03T06:26:22.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.143+0000 7f91bd60c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91b80c95c0 0x7f91b806d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:22.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.143+0000 7f91bd60c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f91b80c95c0 0x7f91b806d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44638/0 (socket says 172.21.15.3:44638) 2023-03-03T06:26:22.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.143+0000 7f91bd60c700 1 -- 172.21.15.3:0/2825498387 learned_addr learned my addr 172.21.15.3:0/2825498387 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:22.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.143+0000 7f91bd60c700 1 -- 172.21.15.3:0/2825498387 >> v1:172.21.15.32:6789/0 conn(0x7f91b815fb60 legacy=0x7f91b8160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:22.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.143+0000 7f91bd60c700 1 -- 172.21.15.3:0/2825498387 >> v1:172.21.15.3:6789/0 conn(0x7f91b80c9a10 legacy=0x7f91b8071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:22.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.143+0000 7f91bd60c700 1 -- 172.21.15.3:0/2825498387 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b80cd4e0 con 0x7f91b80c95c0 2023-03-03T06:26:22.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.147+0000 7f91bd60c700 1 --2- 172.21.15.3:0/2825498387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b80c95c0 0x7f91b806d0a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009910 tx=0x7f91a8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b678ab307e96728 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:22.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.147+0000 7f91bce0b700 1 -- 172.21.15.3:0/2825498387 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a800ba60 con 0x7f91b80c95c0 2023-03-03T06:26:22.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.147+0000 7f91bce0b700 1 -- 172.21.15.3:0/2825498387 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a800bbc0 con 0x7f91b80c95c0 2023-03-03T06:26:22.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.147+0000 7f91bce0b700 1 -- 172.21.15.3:0/2825498387 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a8009e60 con 0x7f91b80c95c0 2023-03-03T06:26:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.147+0000 7f91bee0f700 1 -- 172.21.15.3:0/2825498387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b80c95c0 msgr2=0x7f91b806d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.147+0000 7f91bee0f700 1 --2- 172.21.15.3:0/2825498387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b80c95c0 0x7f91b806d0a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009910 tx=0x7f91a8009c20 comp rx=0 tx=0).stop 2023-03-03T06:26:22.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.147+0000 7f91bee0f700 1 -- 172.21.15.3:0/2825498387 shutdown_connections 2023-03-03T06:26:22.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.147+0000 7f91bee0f700 1 --2- 172.21.15.3:0/2825498387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b80c95c0 0x7f91b806d0a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.147+0000 7f91bee0f700 1 -- 172.21.15.3:0/2825498387 >> 172.21.15.3:0/2825498387 conn(0x7f91b80cfc90 msgr2=0x7f91b80cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:22.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.147+0000 7f91bee0f700 1 -- 172.21.15.3:0/2825498387 shutdown_connections 2023-03-03T06:26:22.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.147+0000 7f91bee0f700 1 -- 172.21.15.3:0/2825498387 wait complete. 2023-03-03T06:26:22.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.151+0000 7f91bee0f700 1 Processor -- start 2023-03-03T06:26:22.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.151+0000 7f91bee0f700 1 -- start start 2023-03-03T06:26:22.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.151+0000 7f91bee0f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b80c95c0 0x7f91b8150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:22.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.151+0000 7f91bee0f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91b80c9a10 0x7f91b80c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:22.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.151+0000 7f91bee0f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815fb60 0x7f91b80c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:22.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.151+0000 7f91bee0f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80b22e0 con 0x7f91b80c95c0 2023-03-03T06:26:22.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.151+0000 7f91bee0f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b8150c90 con 0x7f91b815fb60 2023-03-03T06:26:22.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.151+0000 7f91bee0f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80c39d0 con 0x7f91b80c9a10 2023-03-03T06:26:22.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91bde0d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b80c95c0 0x7f91b8150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:22.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91bde0d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b80c95c0 0x7f91b8150750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44644/0 (socket says 172.21.15.3:44644) 2023-03-03T06:26:22.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91bde0d700 1 -- 172.21.15.3:0/1629989789 learned_addr learned my addr 172.21.15.3:0/1629989789 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:22.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91bd60c700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91b80c9a10 0x7f91b80c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:22.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91be60e700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815fb60 0x7f91b80c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91bde0d700 1 -- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91b80c9a10 msgr2=0x7f91b80c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91bde0d700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91b80c9a10 0x7f91b80c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91bde0d700 1 -- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815fb60 msgr2=0x7f91b80c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91bde0d700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815fb60 0x7f91b80c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.160 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91bde0d700 1 -- 172.21.15.3:0/1629989789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a8009570 con 0x7f91b80c95c0 2023-03-03T06:26:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91bde0d700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b80c95c0 0x7f91b8150750 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f91b4001ed0 tx=0x7f91b40021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91ae7fc700 1 -- 172.21.15.3:0/1629989789 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4003c20 con 0x7f91b80c95c0 2023-03-03T06:26:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91ae7fc700 1 -- 172.21.15.3:0/1629989789 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b4003d80 con 0x7f91b80c95c0 2023-03-03T06:26:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91ae7fc700 1 -- 172.21.15.3:0/1629989789 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b400ab40 con 0x7f91b80c95c0 2023-03-03T06:26:22.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91bee0f700 1 -- 172.21.15.3:0/1629989789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b806cb80 con 0x7f91b80c95c0 2023-03-03T06:26:22.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.155+0000 7f91bee0f700 1 -- 172.21.15.3:0/1629989789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b806c8a0 con 0x7f91b80c95c0 2023-03-03T06:26:22.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.159+0000 7f91bee0f700 1 -- 172.21.15.3:0/1629989789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b80cc850 con 0x7f91b80c95c0 2023-03-03T06:26:22.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.159+0000 7f91ae7fc700 1 -- 172.21.15.3:0/1629989789 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f91b4010050 con 0x7f91b80c95c0 2023-03-03T06:26:22.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.163+0000 7f91ae7fc700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f919c064ee0 0x7f919c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:22.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.163+0000 7f91ae7fc700 1 -- 172.21.15.3:0/1629989789 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f91b4082190 con 0x7f91b80c95c0 2023-03-03T06:26:22.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.163+0000 7f91bd60c700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f919c064ee0 0x7f919c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:22.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.167+0000 7f91bd60c700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f919c064ee0 0x7f919c067390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006f80 tx=0x7f91a8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:22.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.167+0000 7f91ae7fc700 1 -- 172.21.15.3:0/1629989789 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f91b404ff10 con 0x7f91b80c95c0 2023-03-03T06:26:22.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.343+0000 7f91bee0f700 1 -- 172.21.15.3:0/1629989789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f91b806f870 con 0x7f91b80c95c0 2023-03-03T06:26:22.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.347+0000 7f91ae7fc700 1 -- 172.21.15.3:0/1629989789 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f91b4053530 con 0x7f91b80c95c0 2023-03-03T06:26:22.351 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:26:22.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.347+0000 7f91bee0f700 1 -- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f919c064ee0 msgr2=0x7f919c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:22.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.347+0000 7f91bee0f700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f919c064ee0 0x7f919c067390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006f80 tx=0x7f91a8008030 comp rx=0 tx=0).stop 2023-03-03T06:26:22.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.347+0000 7f91bee0f700 1 -- 172.21.15.3:0/1629989789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b80c95c0 msgr2=0x7f91b8150750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:22.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.347+0000 7f91bee0f700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b80c95c0 0x7f91b8150750 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f91b4001ed0 tx=0x7f91b40021e0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.351+0000 7f91bee0f700 1 -- 172.21.15.3:0/1629989789 shutdown_connections 2023-03-03T06:26:22.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.351+0000 7f91bee0f700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f919c064ee0 0x7f919c067390 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.351+0000 7f91bee0f700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91b815fb60 0x7f91b80c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.351+0000 7f91bee0f700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f91b80c9a10 0x7f91b80c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.351+0000 7f91bee0f700 1 --2- 172.21.15.3:0/1629989789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f91b80c95c0 0x7f91b8150750 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.351+0000 7f91bee0f700 1 -- 172.21.15.3:0/1629989789 >> 172.21.15.3:0/1629989789 conn(0x7f91b80cfc90 msgr2=0x7f91b806f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:22.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.351+0000 7f91bee0f700 1 -- 172.21.15.3:0/1629989789 shutdown_connections 2023-03-03T06:26:22.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.351+0000 7f91bee0f700 1 -- 172.21.15.3:0/1629989789 wait complete. 2023-03-03T06:26:22.387 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-03T06:26:22.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.591+0000 7f9cf4bef700 1 Processor -- start 2023-03-03T06:26:22.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.591+0000 7f9cf4bef700 1 -- start start 2023-03-03T06:26:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9cf4bef700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cf015d6e0 0x7f9cf00cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9cf4bef700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9cf00cfd80 0x7f9cf00d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9cf4bef700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf00cf3b0 con 0x7f9cf0156ba0 2023-03-03T06:26:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9cf4bef700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf0156f70 con 0x7f9cf00cfd80 2023-03-03T06:26:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9cf4bef700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf00cd320 con 0x7f9cf015d6e0 2023-03-03T06:26:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9ceffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9cf00cfd80 0x7f9cf00d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9ceffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9cf00cfd80 0x7f9cf00d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44656/0 (socket says 172.21.15.3:44656) 2023-03-03T06:26:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9ceffff700 1 -- 172.21.15.3:0/1367448980 learned_addr learned my addr 172.21.15.3:0/1367448980 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9ceeffd700 1 --2- 172.21.15.3:0/1367448980 >> v2:172.21.15.32:3300/0 conn(0x7f9cf015d6e0 0x7f9cf00cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9ceffff700 1 -- 172.21.15.3:0/1367448980 >> v2:172.21.15.32:3300/0 conn(0x7f9cf015d6e0 msgr2=0x7f9cf00cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9ceffff700 1 --2- 172.21.15.3:0/1367448980 >> v2:172.21.15.32:3300/0 conn(0x7f9cf015d6e0 0x7f9cf00cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9ceffff700 1 -- 172.21.15.3:0/1367448980 >> v1:172.21.15.32:6789/0 conn(0x7f9cf0156ba0 legacy=0x7f9cf0071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9ceffff700 1 -- 172.21.15.3:0/1367448980 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf00cd600 con 0x7f9cf00cfd80 2023-03-03T06:26:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9ceeffd700 1 --2- 172.21.15.3:0/1367448980 >> v2:172.21.15.32:3300/0 conn(0x7f9cf015d6e0 0x7f9cf00cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9ceffff700 1 --2- 172.21.15.3:0/1367448980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cf00cfd80 0x7f9cf00d0b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00b1ca0 tx=0x7f9ce800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3552e326d7d19b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.595+0000 7f9cee7fc700 1 -- 172.21.15.3:0/1367448980 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800e070 con 0x7f9cf00cfd80 2023-03-03T06:26:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.599+0000 7f9cee7fc700 1 -- 172.21.15.3:0/1367448980 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce800b9c0 con 0x7f9cf00cfd80 2023-03-03T06:26:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.599+0000 7f9cee7fc700 1 -- 172.21.15.3:0/1367448980 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800bd70 con 0x7f9cf00cfd80 2023-03-03T06:26:22.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.599+0000 7f9cf4bef700 1 -- 172.21.15.3:0/1367448980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cf00cfd80 msgr2=0x7f9cf00d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:22.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.599+0000 7f9cf4bef700 1 --2- 172.21.15.3:0/1367448980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cf00cfd80 0x7f9cf00d0b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00b1ca0 tx=0x7f9ce800b330 comp rx=0 tx=0).stop 2023-03-03T06:26:22.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.599+0000 7f9cf4bef700 1 -- 172.21.15.3:0/1367448980 shutdown_connections 2023-03-03T06:26:22.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.599+0000 7f9cf4bef700 1 --2- 172.21.15.3:0/1367448980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cf00cfd80 0x7f9cf00d0b30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.599+0000 7f9cf4bef700 1 --2- 172.21.15.3:0/1367448980 >> v2:172.21.15.32:3300/0 conn(0x7f9cf015d6e0 0x7f9cf00cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.599+0000 7f9cf4bef700 1 -- 172.21.15.3:0/1367448980 >> 172.21.15.3:0/1367448980 conn(0x7f9cf006e5c0 msgr2=0x7f9cf006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:22.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.599+0000 7f9cf4bef700 1 -- 172.21.15.3:0/1367448980 shutdown_connections 2023-03-03T06:26:22.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.599+0000 7f9cf4bef700 1 -- 172.21.15.3:0/1367448980 wait complete. 2023-03-03T06:26:22.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.603+0000 7f9cf4bef700 1 Processor -- start 2023-03-03T06:26:22.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.603+0000 7f9cf4bef700 1 -- start start 2023-03-03T06:26:22.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cf4bef700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cf00cfd80 0x7f9cf01551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:22.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cf4bef700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cf0156ba0 0x7f9cf00b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:22.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cf4bef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf015d6e0 0x7f9cf00b8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cf4bef700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00c9d00 con 0x7f9cf00cfd80 2023-03-03T06:26:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cf4bef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00c9fe0 con 0x7f9cf015d6e0 2023-03-03T06:26:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cf4bef700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00c8ef0 con 0x7f9cf0156ba0 2023-03-03T06:26:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cef7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cf00cfd80 0x7f9cf01551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cef7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cf00cfd80 0x7f9cf01551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44662/0 (socket says 172.21.15.3:44662) 2023-03-03T06:26:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cef7fe700 1 -- 172.21.15.3:0/3879452853 learned_addr learned my addr 172.21.15.3:0/3879452853 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9ceeffd700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cf0156ba0 0x7f9cf00b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9ceffff700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf015d6e0 0x7f9cf00b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:22.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cef7fe700 1 -- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cf0156ba0 msgr2=0x7f9cf00b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cef7fe700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cf0156ba0 0x7f9cf00b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cef7fe700 1 -- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf015d6e0 msgr2=0x7f9cf00b8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cef7fe700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf015d6e0 0x7f9cf00b8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cef7fe700 1 -- 172.21.15.3:0/3879452853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce8009b30 con 0x7f9cf00cfd80 2023-03-03T06:26:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9ceffff700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf015d6e0 0x7f9cf00b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:22.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cef7fe700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cf00cfd80 0x7f9cf01551f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000c530 tx=0x7f9ce000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:22.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cd7fff700 1 -- 172.21.15.3:0/3879452853 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00052d0 con 0x7f9cf00cfd80 2023-03-03T06:26:22.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cf4bef700 1 -- 172.21.15.3:0/3879452853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf00cef40 con 0x7f9cf00cfd80 2023-03-03T06:26:22.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cd7fff700 1 -- 172.21.15.3:0/3879452853 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce0005430 con 0x7f9cf00cfd80 2023-03-03T06:26:22.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cd7fff700 1 -- 172.21.15.3:0/3879452853 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00055a0 con 0x7f9cf00cfd80 2023-03-03T06:26:22.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.607+0000 7f9cf4bef700 1 -- 172.21.15.3:0/3879452853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf00cd8e0 con 0x7f9cf00cfd80 2023-03-03T06:26:22.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.611+0000 7f9cf4bef700 1 -- 172.21.15.3:0/3879452853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf00cf0d0 con 0x7f9cf00cfd80 2023-03-03T06:26:22.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.611+0000 7f9cd7fff700 1 -- 172.21.15.3:0/3879452853 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9ce0019030 con 0x7f9cf00cfd80 2023-03-03T06:26:22.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.615+0000 7f9cd7fff700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9cdc064ee0 0x7f9cdc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:22.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.615+0000 7f9cd7fff700 1 -- 172.21.15.3:0/3879452853 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9ce00816b0 con 0x7f9cf00cfd80 2023-03-03T06:26:22.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.615+0000 7f9ceeffd700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9cdc064ee0 0x7f9cdc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:22.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.615+0000 7f9ceeffd700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9cdc064ee0 0x7f9cdc067390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00b1d40 tx=0x7f9ce4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:22.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.619+0000 7f9cd7fff700 1 -- 172.21.15.3:0/3879452853 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9ce004f3b0 con 0x7f9cf00cfd80 2023-03-03T06:26:22.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.799+0000 7f9cf4bef700 1 -- 172.21.15.3:0/3879452853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9cf0073e90 con 0x7f9cf00cfd80 2023-03-03T06:26:22.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.799+0000 7f9cd7fff700 1 -- 172.21.15.3:0/3879452853 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ce00529d0 con 0x7f9cf00cfd80 2023-03-03T06:26:22.804 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.803+0000 7f9cf4bef700 1 -- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9cdc064ee0 msgr2=0x7f9cdc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.803+0000 7f9cf4bef700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9cdc064ee0 0x7f9cdc067390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00b1d40 tx=0x7f9ce4008040 comp rx=0 tx=0).stop 2023-03-03T06:26:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.803+0000 7f9cf4bef700 1 -- 172.21.15.3:0/3879452853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cf00cfd80 msgr2=0x7f9cf01551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.803+0000 7f9cf4bef700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cf00cfd80 0x7f9cf01551f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000c530 tx=0x7f9ce000c8f0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.803+0000 7f9cf4bef700 1 -- 172.21.15.3:0/3879452853 shutdown_connections 2023-03-03T06:26:22.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.803+0000 7f9cf4bef700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9cdc064ee0 0x7f9cdc067390 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.803+0000 7f9cf4bef700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cf015d6e0 0x7f9cf00b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.803+0000 7f9cf4bef700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9cf0156ba0 0x7f9cf00b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.803+0000 7f9cf4bef700 1 --2- 172.21.15.3:0/3879452853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9cf00cfd80 0x7f9cf01551f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:22.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.803+0000 7f9cf4bef700 1 -- 172.21.15.3:0/3879452853 >> 172.21.15.3:0/3879452853 conn(0x7f9cf006e5c0 msgr2=0x7f9cf00d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:22.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.803+0000 7f9cf4bef700 1 -- 172.21.15.3:0/3879452853 shutdown_connections 2023-03-03T06:26:22.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:22.803+0000 7f9cf4bef700 1 -- 172.21.15.3:0/3879452853 wait complete. 2023-03-03T06:26:22.843 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-03T06:26:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.051+0000 7f219af28700 1 Processor -- start 2023-03-03T06:26:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.051+0000 7f219af28700 1 -- start start 2023-03-03T06:26:23.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f219af28700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21940cc9b0 0x7f21940cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f219af28700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21940b19e0 0x7f219406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f219af28700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f219406e6f0 con 0x7f21941609b0 2023-03-03T06:26:23.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f219af28700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2194160f50 con 0x7f21940b19e0 2023-03-03T06:26:23.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f219af28700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21940cf5f0 con 0x7f21940cc9b0 2023-03-03T06:26:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f219a727700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21940b19e0 0x7f219406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f2199f26700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21940cc9b0 0x7f21940cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f2199725700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f21941609b0 0x7f21940b12d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:46022/0 (socket says 172.21.15.3:46022) 2023-03-03T06:26:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f2199725700 1 -- 172.21.15.3:0/3308435331 learned_addr learned my addr 172.21.15.3:0/3308435331 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f2198f24700 1 -- 172.21.15.3:0/3308435331 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660176551 0 0) 0x7f219406e6f0 con 0x7f21941609b0 2023-03-03T06:26:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f2198f24700 1 -- 172.21.15.3:0/3308435331 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21800035d0 con 0x7f21941609b0 2023-03-03T06:26:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f2199f26700 1 -- 172.21.15.3:0/3308435331 >> v2:172.21.15.3:3300/0 conn(0x7f21940b19e0 msgr2=0x7f219406eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f2199f26700 1 --2- 172.21.15.3:0/3308435331 >> v2:172.21.15.3:3300/0 conn(0x7f21940b19e0 0x7f219406eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f2199f26700 1 -- 172.21.15.3:0/3308435331 >> v1:172.21.15.32:6789/0 conn(0x7f21941609b0 legacy=0x7f21940b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f2199f26700 1 -- 172.21.15.3:0/3308435331 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219415d880 con 0x7f21940cc9b0 2023-03-03T06:26:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.055+0000 7f219a727700 1 --2- 172.21.15.3:0/3308435331 >> v2:172.21.15.3:3300/0 conn(0x7f21940b19e0 0x7f219406eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.059+0000 7f2199f26700 1 --2- 172.21.15.3:0/3308435331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21940cc9b0 0x7f21940cf0b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2190009de0 tx=0x7f219000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef57b1de8f620d75 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:23.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.059+0000 7f2198f24700 1 -- 172.21.15.3:0/3308435331 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219000e070 con 0x7f21940cc9b0 2023-03-03T06:26:23.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.059+0000 7f2198f24700 1 -- 172.21.15.3:0/3308435331 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219000b910 con 0x7f21940cc9b0 2023-03-03T06:26:23.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.059+0000 7f2198f24700 1 -- 172.21.15.3:0/3308435331 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219000bcc0 con 0x7f21940cc9b0 2023-03-03T06:26:23.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.059+0000 7f219af28700 1 -- 172.21.15.3:0/3308435331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21940cc9b0 msgr2=0x7f21940cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.059+0000 7f219af28700 1 --2- 172.21.15.3:0/3308435331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21940cc9b0 0x7f21940cf0b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2190009de0 tx=0x7f219000b330 comp rx=0 tx=0).stop 2023-03-03T06:26:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.059+0000 7f219af28700 1 -- 172.21.15.3:0/3308435331 shutdown_connections 2023-03-03T06:26:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.059+0000 7f219af28700 1 --2- 172.21.15.3:0/3308435331 >> v2:172.21.15.3:3300/0 conn(0x7f21940b19e0 0x7f219406eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.059+0000 7f219af28700 1 --2- 172.21.15.3:0/3308435331 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21940cc9b0 0x7f21940cf0b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.059+0000 7f219af28700 1 -- 172.21.15.3:0/3308435331 >> 172.21.15.3:0/3308435331 conn(0x7f2194069d70 msgr2=0x7f219406a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.063+0000 7f219af28700 1 -- 172.21.15.3:0/3308435331 shutdown_connections 2023-03-03T06:26:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.063+0000 7f219af28700 1 -- 172.21.15.3:0/3308435331 wait complete. 2023-03-03T06:26:23.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.067+0000 7f219af28700 1 Processor -- start 2023-03-03T06:26:23.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.067+0000 7f219af28700 1 -- start start 2023-03-03T06:26:23.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.067+0000 7f219af28700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21940b19e0 0x7f21941f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.067+0000 7f219af28700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21940cc9b0 0x7f21940b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.067+0000 7f219af28700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21941609b0 0x7f219414cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.067+0000 7f219af28700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21941f2760 con 0x7f21940cc9b0 2023-03-03T06:26:23.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.067+0000 7f219af28700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21940c4340 con 0x7f21941609b0 2023-03-03T06:26:23.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.067+0000 7f219af28700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21940c4620 con 0x7f21940b19e0 2023-03-03T06:26:23.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.067+0000 7f2199725700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21940cc9b0 0x7f21940b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.067+0000 7f2199725700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21940cc9b0 0x7f21940b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44674/0 (socket says 172.21.15.3:44674) 2023-03-03T06:26:23.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.067+0000 7f2199725700 1 -- 172.21.15.3:0/4241001980 learned_addr learned my addr 172.21.15.3:0/4241001980 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:23.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.067+0000 7f219a727700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21941609b0 0x7f219414cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.067+0000 7f2199f26700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21940b19e0 0x7f21941f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.071+0000 7f2199725700 1 -- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21940b19e0 msgr2=0x7f21941f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.071+0000 7f2199725700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21940b19e0 0x7f21941f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.071+0000 7f2199725700 1 -- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21941609b0 msgr2=0x7f219414cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.071+0000 7f2199725700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21941609b0 0x7f219414cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.071+0000 7f2199725700 1 -- 172.21.15.3:0/4241001980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c0094b0 con 0x7f21940cc9b0 2023-03-03T06:26:23.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.071+0000 7f2199725700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21940cc9b0 0x7f21940b4fa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f218400c4d0 tx=0x7f218400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:23.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.071+0000 7f218a7fc700 1 -- 172.21.15.3:0/4241001980 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2184005d90 con 0x7f21940cc9b0 2023-03-03T06:26:23.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.071+0000 7f218a7fc700 1 -- 172.21.15.3:0/4241001980 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21840070d0 con 0x7f21940cc9b0 2023-03-03T06:26:23.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.071+0000 7f218a7fc700 1 -- 172.21.15.3:0/4241001980 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2184007480 con 0x7f21940cc9b0 2023-03-03T06:26:23.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.071+0000 7f219af28700 1 -- 172.21.15.3:0/4241001980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2190009a70 con 0x7f21940cc9b0 2023-03-03T06:26:23.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.071+0000 7f219af28700 1 -- 172.21.15.3:0/4241001980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21940cd530 con 0x7f21940cc9b0 2023-03-03T06:26:23.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.071+0000 7f219af28700 1 -- 172.21.15.3:0/4241001980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21940b20c0 con 0x7f21940cc9b0 2023-03-03T06:26:23.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.075+0000 7f218a7fc700 1 -- 172.21.15.3:0/4241001980 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f2184019030 con 0x7f21940cc9b0 2023-03-03T06:26:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.079+0000 7f218a7fc700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2178064ee0 0x7f2178067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.079+0000 7f218a7fc700 1 -- 172.21.15.3:0/4241001980 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2184081520 con 0x7f21940cc9b0 2023-03-03T06:26:23.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.079+0000 7f2199f26700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2178064ee0 0x7f2178067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.079+0000 7f2199f26700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2178064ee0 0x7f2178067390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2190014040 tx=0x7f2190011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:23.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.083+0000 7f218a7fc700 1 -- 172.21.15.3:0/4241001980 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f218404f2a0 con 0x7f21940cc9b0 2023-03-03T06:26:23.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.255+0000 7f219af28700 1 -- 172.21.15.3:0/4241001980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f21940cd810 con 0x7f21940cc9b0 2023-03-03T06:26:23.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.255+0000 7f218a7fc700 1 -- 172.21.15.3:0/4241001980 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f21840528c0 con 0x7f21940cc9b0 2023-03-03T06:26:23.261 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-03-03T06:26:23.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.263+0000 7f219af28700 1 -- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2178064ee0 msgr2=0x7f2178067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.263+0000 7f219af28700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2178064ee0 0x7f2178067390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2190014040 tx=0x7f2190011000 comp rx=0 tx=0).stop 2023-03-03T06:26:23.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.263+0000 7f219af28700 1 -- 172.21.15.3:0/4241001980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21940cc9b0 msgr2=0x7f21940b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.263+0000 7f219af28700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21940cc9b0 0x7f21940b4fa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f218400c4d0 tx=0x7f218400c7e0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.263+0000 7f219af28700 1 -- 172.21.15.3:0/4241001980 shutdown_connections 2023-03-03T06:26:23.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.263+0000 7f219af28700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2178064ee0 0x7f2178067390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.263+0000 7f219af28700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21941609b0 0x7f219414cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.263+0000 7f219af28700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21940cc9b0 0x7f21940b4fa0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.263+0000 7f219af28700 1 --2- 172.21.15.3:0/4241001980 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21940b19e0 0x7f21941f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.263+0000 7f219af28700 1 -- 172.21.15.3:0/4241001980 >> 172.21.15.3:0/4241001980 conn(0x7f2194069d70 msgr2=0x7f21940cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:23.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.263+0000 7f219af28700 1 -- 172.21.15.3:0/4241001980 shutdown_connections 2023-03-03T06:26:23.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.263+0000 7f219af28700 1 -- 172.21.15.3:0/4241001980 wait complete. 2023-03-03T06:26:23.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-03T06:26:23.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.507+0000 7fc0c4849700 1 Processor -- start 2023-03-03T06:26:23.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.507+0000 7fc0c4849700 1 -- start start 2023-03-03T06:26:23.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.511+0000 7fc0c4849700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0c00b0ed0 0x7fc0c006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.511+0000 7fc0c4849700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0c00cb520 0x7fc0c015d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.511+0000 7fc0c4849700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c015d940 con 0x7fc0c015c370 2023-03-03T06:26:23.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.511+0000 7fc0c4849700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c00ce520 con 0x7fc0c00cb520 2023-03-03T06:26:23.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.511+0000 7fc0c4849700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00ccbc0 con 0x7fc0c00b0ed0 2023-03-03T06:26:23.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.511+0000 7fc0bed9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc0c015c370 0x7fc0c00b07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:46032/0 (socket says 172.21.15.3:46032) 2023-03-03T06:26:23.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.511+0000 7fc0bed9d700 1 -- 172.21.15.3:0/764130092 learned_addr learned my addr 172.21.15.3:0/764130092 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:23.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.511+0000 7fc0be59c700 1 --2- 172.21.15.3:0/764130092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0c00b0ed0 0x7fc0c006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.511+0000 7fc0bf59e700 1 --2- 172.21.15.3:0/764130092 >> v2:172.21.15.32:3300/0 conn(0x7fc0c00cb520 0x7fc0c015d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.511+0000 7fc0bdd9b700 1 -- 172.21.15.3:0/764130092 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384592349 0 0) 0x7fc0b0001110 con 0x7fc0c015c370 2023-03-03T06:26:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0bdd9b700 1 -- 172.21.15.3:0/764130092 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a80035d0 con 0x7fc0c015c370 2023-03-03T06:26:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0be59c700 1 -- 172.21.15.3:0/764130092 >> v2:172.21.15.32:3300/0 conn(0x7fc0c00cb520 msgr2=0x7fc0c015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0be59c700 1 --2- 172.21.15.3:0/764130092 >> v2:172.21.15.32:3300/0 conn(0x7fc0c00cb520 0x7fc0c015d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0be59c700 1 -- 172.21.15.3:0/764130092 >> v1:172.21.15.32:6789/0 conn(0x7fc0c015c370 legacy=0x7fc0c00b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0be59c700 1 -- 172.21.15.3:0/764130092 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c00cbae0 con 0x7fc0c00b0ed0 2023-03-03T06:26:23.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0be59c700 1 --2- 172.21.15.3:0/764130092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0c00b0ed0 0x7fc0c006c090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4009850 tx=0x7fc0b4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aba0b676d9ea614 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:23.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0bdd9b700 1 -- 172.21.15.3:0/764130092 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4011830 con 0x7fc0c00b0ed0 2023-03-03T06:26:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0bdd9b700 1 -- 172.21.15.3:0/764130092 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b4011990 con 0x7fc0c00b0ed0 2023-03-03T06:26:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0bdd9b700 1 -- 172.21.15.3:0/764130092 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4011d40 con 0x7fc0c00b0ed0 2023-03-03T06:26:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0c4849700 1 -- 172.21.15.3:0/764130092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0c00b0ed0 msgr2=0x7fc0c006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0c4849700 1 --2- 172.21.15.3:0/764130092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0c00b0ed0 0x7fc0c006c090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4009850 tx=0x7fc0b4009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0c4849700 1 -- 172.21.15.3:0/764130092 shutdown_connections 2023-03-03T06:26:23.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0c4849700 1 --2- 172.21.15.3:0/764130092 >> v2:172.21.15.32:3300/0 conn(0x7fc0c00cb520 0x7fc0c015d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0c4849700 1 --2- 172.21.15.3:0/764130092 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0c00b0ed0 0x7fc0c006c090 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0c4849700 1 -- 172.21.15.3:0/764130092 >> 172.21.15.3:0/764130092 conn(0x7fc0c00607c0 msgr2=0x7fc0c0060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:23.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0c4849700 1 -- 172.21.15.3:0/764130092 shutdown_connections 2023-03-03T06:26:23.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.515+0000 7fc0c4849700 1 -- 172.21.15.3:0/764130092 wait complete. 2023-03-03T06:26:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.519+0000 7fc0c4849700 1 Processor -- start 2023-03-03T06:26:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.519+0000 7fc0c4849700 1 -- start start 2023-03-03T06:26:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.519+0000 7fc0c4849700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c00b0ed0 0x7fc0c014dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0c4849700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0c00cb520 0x7fc0c00c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0c4849700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0c015c370 0x7fc0c00af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0c4849700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c01f60a0 con 0x7fc0c015c370 2023-03-03T06:26:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0c4849700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c01f5420 con 0x7fc0c00b0ed0 2023-03-03T06:26:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0c4849700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c01f5700 con 0x7fc0c00cb520 2023-03-03T06:26:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0bf59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0c015c370 0x7fc0c00af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0bf59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0c015c370 0x7fc0c00af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44682/0 (socket says 172.21.15.3:44682) 2023-03-03T06:26:23.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0bf59e700 1 -- 172.21.15.3:0/113648791 learned_addr learned my addr 172.21.15.3:0/113648791 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:23.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0bed9d700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c00b0ed0 0x7fc0c014dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0be59c700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0c00cb520 0x7fc0c00c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0bf59e700 1 -- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0c00cb520 msgr2=0x7fc0c00c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0bf59e700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0c00cb520 0x7fc0c00c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0bf59e700 1 -- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c00b0ed0 msgr2=0x7fc0c014dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0bf59e700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c00b0ed0 0x7fc0c014dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0bf59e700 1 -- 172.21.15.3:0/113648791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b40094b0 con 0x7fc0c015c370 2023-03-03T06:26:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0bed9d700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c00b0ed0 0x7fc0c014dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:23.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0bf59e700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0c015c370 0x7fc0c00af180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc0b800efc0 tx=0x7fc0b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:23.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0a77fe700 1 -- 172.21.15.3:0/113648791 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8009ea0 con 0x7fc0c015c370 2023-03-03T06:26:23.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0a77fe700 1 -- 172.21.15.3:0/113648791 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b800f040 con 0x7fc0c015c370 2023-03-03T06:26:23.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0a77fe700 1 -- 172.21.15.3:0/113648791 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b800ced0 con 0x7fc0c015c370 2023-03-03T06:26:23.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0c4849700 1 -- 172.21.15.3:0/113648791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c00cc7e0 con 0x7fc0c015c370 2023-03-03T06:26:23.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.523+0000 7fc0c4849700 1 -- 172.21.15.3:0/113648791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c00cc500 con 0x7fc0c015c370 2023-03-03T06:26:23.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.527+0000 7fc0c4849700 1 -- 172.21.15.3:0/113648791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c015c740 con 0x7fc0c015c370 2023-03-03T06:26:23.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.527+0000 7fc0a77fe700 1 -- 172.21.15.3:0/113648791 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc0b8014480 con 0x7fc0c015c370 2023-03-03T06:26:23.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.531+0000 7fc0a77fe700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc0ac064ee0 0x7fc0ac067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.531+0000 7fc0a77fe700 1 -- 172.21.15.3:0/113648791 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc0b8082b00 con 0x7fc0c015c370 2023-03-03T06:26:23.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.531+0000 7fc0bed9d700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc0ac064ee0 0x7fc0ac067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.531+0000 7fc0bed9d700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc0ac064ee0 0x7fc0ac067390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0007fd0 tx=0x7fc0b0007e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:23.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.535+0000 7fc0a77fe700 1 -- 172.21.15.3:0/113648791 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc0b8050910 con 0x7fc0c015c370 2023-03-03T06:26:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.711+0000 7fc0c4849700 1 -- 172.21.15.3:0/113648791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc0c01f61e0 con 0x7fc0c015c370 2023-03-03T06:26:23.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.711+0000 7fc0a77fe700 1 -- 172.21.15.3:0/113648791 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc0c01f61e0 con 0x7fc0c015c370 2023-03-03T06:26:23.715 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:23.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.715+0000 7fc0c4849700 1 -- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc0ac064ee0 msgr2=0x7fc0ac067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.715+0000 7fc0c4849700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc0ac064ee0 0x7fc0ac067390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0007fd0 tx=0x7fc0b0007e70 comp rx=0 tx=0).stop 2023-03-03T06:26:23.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.715+0000 7fc0c4849700 1 -- 172.21.15.3:0/113648791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0c015c370 msgr2=0x7fc0c00af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.715+0000 7fc0c4849700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0c015c370 0x7fc0c00af180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc0b800efc0 tx=0x7fc0b800c3a0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.719+0000 7fc0c4849700 1 -- 172.21.15.3:0/113648791 shutdown_connections 2023-03-03T06:26:23.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.719+0000 7fc0c4849700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc0ac064ee0 0x7fc0ac067390 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.719+0000 7fc0c4849700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0c015c370 0x7fc0c00af180 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.719+0000 7fc0c4849700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc0c00cb520 0x7fc0c00c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.719+0000 7fc0c4849700 1 --2- 172.21.15.3:0/113648791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0c00b0ed0 0x7fc0c014dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.719+0000 7fc0c4849700 1 -- 172.21.15.3:0/113648791 >> 172.21.15.3:0/113648791 conn(0x7fc0c00607c0 msgr2=0x7fc0c00ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.719+0000 7fc0c4849700 1 -- 172.21.15.3:0/113648791 shutdown_connections 2023-03-03T06:26:23.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.719+0000 7fc0c4849700 1 -- 172.21.15.3:0/113648791 wait complete. 2023-03-03T06:26:23.754 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-03T06:26:23.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.959+0000 7f588575d700 1 Processor -- start 2023-03-03T06:26:23.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.959+0000 7f588575d700 1 -- start start 2023-03-03T06:26:23.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.963+0000 7f588575d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58800cb890 0x7f588015e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.963+0000 7f588575d700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58800c9a20 con 0x7f58800adf20 2023-03-03T06:26:23.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.963+0000 7f588575d700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58800c9fe0 con 0x7f588015dec0 2023-03-03T06:26:23.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.963+0000 7f588575d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58800ae2f0 con 0x7f58800cb890 2023-03-03T06:26:23.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.963+0000 7f587ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f58800adf20 0x7f58800b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53644/0 (socket says 172.21.15.3:53644) 2023-03-03T06:26:23.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.963+0000 7f587ffff700 1 -- 172.21.15.3:0/3292026610 learned_addr learned my addr 172.21.15.3:0/3292026610 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:23.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.963+0000 7f5884f5c700 1 --2- 172.21.15.3:0/3292026610 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58800cb890 0x7f588015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f587f7fe700 1 -- 172.21.15.3:0/3292026610 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860058765 0 0) 0x7f58800c9a20 con 0x7f58800adf20 2023-03-03T06:26:23.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f587f7fe700 1 -- 172.21.15.3:0/3292026610 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f586c0035d0 con 0x7f58800adf20 2023-03-03T06:26:23.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f5884f5c700 1 -- 172.21.15.3:0/3292026610 >> v1:172.21.15.32:6789/0 conn(0x7f588015dec0 legacy=0x7f58800cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f587f7fe700 1 -- 172.21.15.3:0/3292026610 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700594672 0 0) 0x7f58800c9fe0 con 0x7f588015dec0 2023-03-03T06:26:23.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f5884f5c700 1 -- 172.21.15.3:0/3292026610 >> v1:172.21.15.3:6789/0 conn(0x7f58800adf20 legacy=0x7f58800b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f5884f5c700 1 -- 172.21.15.3:0/3292026610 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588015a7d0 con 0x7f58800cb890 2023-03-03T06:26:23.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f5884f5c700 1 --2- 172.21.15.3:0/3292026610 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58800cb890 0x7f588015e310 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5878009dc0 tx=0x7f587800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4818130b28b572ba server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f587f7fe700 1 -- 172.21.15.3:0/3292026610 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587800e070 con 0x7f58800cb890 2023-03-03T06:26:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f587f7fe700 1 -- 172.21.15.3:0/3292026610 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587800bb20 con 0x7f58800cb890 2023-03-03T06:26:23.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f587f7fe700 1 -- 172.21.15.3:0/3292026610 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587800bed0 con 0x7f58800cb890 2023-03-03T06:26:23.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f588575d700 1 -- 172.21.15.3:0/3292026610 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58800cb890 msgr2=0x7f588015e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f588575d700 1 --2- 172.21.15.3:0/3292026610 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58800cb890 0x7f588015e310 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5878009dc0 tx=0x7f587800b330 comp rx=0 tx=0).stop 2023-03-03T06:26:23.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f588575d700 1 -- 172.21.15.3:0/3292026610 shutdown_connections 2023-03-03T06:26:23.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f588575d700 1 --2- 172.21.15.3:0/3292026610 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58800cb890 0x7f588015e310 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.967+0000 7f588575d700 1 -- 172.21.15.3:0/3292026610 >> 172.21.15.3:0/3292026610 conn(0x7f5880070a30 msgr2=0x7f5880070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:23.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.971+0000 7f588575d700 1 -- 172.21.15.3:0/3292026610 shutdown_connections 2023-03-03T06:26:23.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.971+0000 7f588575d700 1 -- 172.21.15.3:0/3292026610 wait complete. 2023-03-03T06:26:23.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.971+0000 7f588575d700 1 Processor -- start 2023-03-03T06:26:23.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.971+0000 7f588575d700 1 -- start start 2023-03-03T06:26:23.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f588575d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800adf20 0x7f588015c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f588575d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58800cb890 0x7f588006c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f588575d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f588015dec0 0x7f588015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f588575d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5880152eb0 con 0x7f58800adf20 2023-03-03T06:26:23.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f588575d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5880151740 con 0x7f588015dec0 2023-03-03T06:26:23.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f588575d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5880153150 con 0x7f58800cb890 2023-03-03T06:26:23.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f587ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800adf20 0x7f588015c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f587ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800adf20 0x7f588015c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44698/0 (socket says 172.21.15.3:44698) 2023-03-03T06:26:23.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f587ffff700 1 -- 172.21.15.3:0/2221433839 learned_addr learned my addr 172.21.15.3:0/2221433839 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:23.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f5877fff700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58800cb890 0x7f588006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f5884f5c700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f588015dec0 0x7f588015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f587ffff700 1 -- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58800cb890 msgr2=0x7f588006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f587ffff700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58800cb890 0x7f588006c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f587ffff700 1 -- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f588015dec0 msgr2=0x7f588015a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:23.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f587ffff700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f588015dec0 0x7f588015a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:23.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f587ffff700 1 -- 172.21.15.3:0/2221433839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5878009a70 con 0x7f58800adf20 2023-03-03T06:26:23.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f587ffff700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800adf20 0x7f588015c9b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f587000c4f0 tx=0x7f587000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:23.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f587d7fa700 1 -- 172.21.15.3:0/2221433839 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870005d90 con 0x7f58800adf20 2023-03-03T06:26:23.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f587d7fa700 1 -- 172.21.15.3:0/2221433839 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58700070d0 con 0x7f58800adf20 2023-03-03T06:26:23.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f588575d700 1 -- 172.21.15.3:0/2221433839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588015ebf0 con 0x7f58800adf20 2023-03-03T06:26:23.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f587d7fa700 1 -- 172.21.15.3:0/2221433839 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870007440 con 0x7f58800adf20 2023-03-03T06:26:23.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.975+0000 7f588575d700 1 -- 172.21.15.3:0/2221433839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588015e910 con 0x7f58800adf20 2023-03-03T06:26:23.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.979+0000 7f588575d700 1 -- 172.21.15.3:0/2221433839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58800c9c70 con 0x7f58800adf20 2023-03-03T06:26:23.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.979+0000 7f587d7fa700 1 -- 172.21.15.3:0/2221433839 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5870019030 con 0x7f58800adf20 2023-03-03T06:26:23.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.983+0000 7f587d7fa700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5860064f30 0x7f58600673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:23.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.983+0000 7f587d7fa700 1 -- 172.21.15.3:0/2221433839 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f58700815a0 con 0x7f58800adf20 2023-03-03T06:26:23.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.983+0000 7f5877fff700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5860064f30 0x7f58600673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:23.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.983+0000 7f5877fff700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5860064f30 0x7f58600673e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f588006db40 tx=0x7f5868009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:23.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:23.987+0000 7f587d7fa700 1 -- 172.21.15.3:0/2221433839 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f587004a400 con 0x7f58800adf20 2023-03-03T06:26:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.179+0000 7f588575d700 1 -- 172.21.15.3:0/2221433839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5880073e90 con 0x7f58800adf20 2023-03-03T06:26:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.183+0000 7f587d7fa700 1 -- 172.21.15.3:0/2221433839 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f587004f450 con 0x7f58800adf20 2023-03-03T06:26:24.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.183+0000 7f588575d700 1 -- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5860064f30 msgr2=0x7f58600673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:24.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.183+0000 7f588575d700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5860064f30 0x7f58600673e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f588006db40 tx=0x7f5868009040 comp rx=0 tx=0).stop 2023-03-03T06:26:24.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.183+0000 7f588575d700 1 -- 172.21.15.3:0/2221433839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800adf20 msgr2=0x7f588015c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:24.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.183+0000 7f588575d700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800adf20 0x7f588015c9b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f587000c4f0 tx=0x7f587000c8b0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.183+0000 7f588575d700 1 -- 172.21.15.3:0/2221433839 shutdown_connections 2023-03-03T06:26:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.183+0000 7f588575d700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5860064f30 0x7f58600673e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.183+0000 7f588575d700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f588015dec0 0x7f588015a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.183+0000 7f588575d700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f58800cb890 0x7f588006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.183+0000 7f588575d700 1 --2- 172.21.15.3:0/2221433839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58800adf20 0x7f588015c9b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.183+0000 7f588575d700 1 -- 172.21.15.3:0/2221433839 >> 172.21.15.3:0/2221433839 conn(0x7f5880070a30 msgr2=0x7f58800ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.183+0000 7f588575d700 1 -- 172.21.15.3:0/2221433839 shutdown_connections 2023-03-03T06:26:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.183+0000 7f588575d700 1 -- 172.21.15.3:0/2221433839 wait complete. 2023-03-03T06:26:24.192 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2023-03-03T06:26:24.223 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-03-03T06:26:24.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:24.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-03-03T06:26:24.332 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-03-03T06:26:24.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:24.333 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-03-03T06:26:24.364 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:26:24.365 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:24.386 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:26:24.387 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:24.451 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:26:24.452 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:24.514 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:24.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.755+0000 7f5d016c4700 1 Processor -- start 2023-03-03T06:26:24.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.755+0000 7f5d016c4700 1 -- start start 2023-03-03T06:26:24.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5d016c4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5cfc15c3d0 0x7f5cfc15c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:24.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5d016c4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cfc0c9fc0 0x7f5cfc0cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5d016c4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cfc15d940 con 0x7f5cfc0cb770 2023-03-03T06:26:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5d016c4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cfc15cdc0 con 0x7f5cfc15c3d0 2023-03-03T06:26:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5d016c4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cfc15d0a0 con 0x7f5cfc0c9fc0 2023-03-03T06:26:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cf3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5cfc0cb770 0x7f5cfc0c98b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53656/0 (socket says 172.21.15.3:53656) 2023-03-03T06:26:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cf3fff700 1 -- 172.21.15.3:0/1336753595 learned_addr learned my addr 172.21.15.3:0/1336753595 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfbfff700 1 --2- 172.21.15.3:0/1336753595 >> v2:172.21.15.3:3300/0 conn(0x7f5cfc15c3d0 0x7f5cfc15c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:24.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5d00ec3700 1 --2- 172.21.15.3:0/1336753595 >> v2:172.21.15.32:3300/0 conn(0x7f5cfc0c9fc0 0x7f5cfc0cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226168173 0 0) 0x7f5cfc15d940 con 0x7f5cfc0cb770 2023-03-03T06:26:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ce80035d0 con 0x7f5cfc0cb770 2023-03-03T06:26:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2052907852 0 0) 0x7f5ce80035d0 con 0x7f5cfc0cb770 2023-03-03T06:26:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 >> v2:172.21.15.32:3300/0 conn(0x7f5cfc0c9fc0 msgr2=0x7f5cfc0cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 --2- 172.21.15.3:0/1336753595 >> v2:172.21.15.32:3300/0 conn(0x7f5cfc0c9fc0 0x7f5cfc0cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 >> v2:172.21.15.3:3300/0 conn(0x7f5cfc15c3d0 msgr2=0x7f5cfc15c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 --2- 172.21.15.3:0/1336753595 >> v2:172.21.15.3:3300/0 conn(0x7f5cfc15c3d0 0x7f5cfc15c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc15d380 con 0x7f5cfc0cb770 2023-03-03T06:26:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f5ce4001670 con 0x7f5cfc0cb770 2023-03-03T06:26:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 >> v1:172.21.15.3:6789/0 conn(0x7f5cfc0cb770 legacy=0x7f5cfc0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 --2- 172.21.15.3:0/1336753595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce8003a20 0x7f5ce8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 --2- 172.21.15.3:0/1336753595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce8004870 0x7f5ce8004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 --2- 172.21.15.3:0/1336753595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce8005220 0x7f5ce800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc15cdc0 con 0x7f5ce8005220 2023-03-03T06:26:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc15d0a0 con 0x7f5ce8003a20 2023-03-03T06:26:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce800bfa0 con 0x7f5ce8004870 2023-03-03T06:26:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5d00ec3700 1 --2- 172.21.15.3:0/1336753595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce8005220 0x7f5ce800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cf3fff700 1 --2- 172.21.15.3:0/1336753595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce8004870 0x7f5ce8004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.759+0000 7f5cfbfff700 1 --2- 172.21.15.3:0/1336753595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce8003a20 0x7f5ce8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:24.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5cf3fff700 1 -- 172.21.15.3:0/1336753595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce8003a20 msgr2=0x7f5ce8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:24.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5cf3fff700 1 --2- 172.21.15.3:0/1336753595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce8003a20 0x7f5ce8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5cf3fff700 1 -- 172.21.15.3:0/1336753595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce8005220 msgr2=0x7f5ce800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5cf3fff700 1 --2- 172.21.15.3:0/1336753595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce8005220 0x7f5ce800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5cf3fff700 1 -- 172.21.15.3:0/1336753595 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce80035d0 con 0x7f5ce8004870 2023-03-03T06:26:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5d00ec3700 1 -- 172.21.15.3:0/1336753595 reap_dead start 2023-03-03T06:26:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5cf3fff700 1 --2- 172.21.15.3:0/1336753595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce8004870 0x7f5ce8004ce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400c320 tx=0x7f5ce80058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2063c63a6b7230f server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce400cde0 con 0x7f5ce8004870 2023-03-03T06:26:24.769 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce4005980 con 0x7f5ce8004870 2023-03-03T06:26:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5cfb7fe700 1 -- 172.21.15.3:0/1336753595 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4005d30 con 0x7f5ce8004870 2023-03-03T06:26:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5d016c4700 1 -- 172.21.15.3:0/1336753595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce8004870 msgr2=0x7f5ce8004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5d016c4700 1 --2- 172.21.15.3:0/1336753595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce8004870 0x7f5ce8004ce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400c320 tx=0x7f5ce80058a0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5d016c4700 1 -- 172.21.15.3:0/1336753595 shutdown_connections 2023-03-03T06:26:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5d016c4700 1 --2- 172.21.15.3:0/1336753595 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce8004870 0x7f5ce8004ce0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5d016c4700 1 -- 172.21.15.3:0/1336753595 >> 172.21.15.3:0/1336753595 conn(0x7f5cfc070a30 msgr2=0x7f5cfc070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:24.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5d016c4700 1 -- 172.21.15.3:0/1336753595 shutdown_connections 2023-03-03T06:26:24.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.763+0000 7f5d016c4700 1 -- 172.21.15.3:0/1336753595 wait complete. 2023-03-03T06:26:24.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.767+0000 7f5d016c4700 1 Processor -- start 2023-03-03T06:26:24.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.767+0000 7f5d016c4700 1 -- start start 2023-03-03T06:26:24.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d016c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce8004870 0x7f5cfc1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d016c4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cfc155a30 0x7f5cfc0ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d016c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cfc1553d0 0x7f5cfc15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d016c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc158790 con 0x7f5cfc1553d0 2023-03-03T06:26:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d016c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc155840 con 0x7f5ce8004870 2023-03-03T06:26:24.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d016c4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc1589e0 con 0x7f5cfc155a30 2023-03-03T06:26:24.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d00ec3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cfc1553d0 0x7f5cfc15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:24.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d00ec3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cfc1553d0 0x7f5cfc15a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44730/0 (socket says 172.21.15.3:44730) 2023-03-03T06:26:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d00ec3700 1 -- 172.21.15.3:0/3293564360 learned_addr learned my addr 172.21.15.3:0/3293564360 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5cf3fff700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cfc155a30 0x7f5cfc0ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5cfbfff700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce8004870 0x7f5cfc1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d00ec3700 1 -- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cfc155a30 msgr2=0x7f5cfc0ab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d00ec3700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cfc155a30 0x7f5cfc0ab2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d00ec3700 1 -- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce8004870 msgr2=0x7f5cfc1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:24.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d00ec3700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce8004870 0x7f5cfc1534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d00ec3700 1 -- 172.21.15.3:0/3293564360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc0cbbe0 con 0x7f5cfc1553d0 2023-03-03T06:26:24.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5cf3fff700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cfc155a30 0x7f5cfc0ab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:24.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d00ec3700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cfc1553d0 0x7f5cfc15a2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0ca3b0 tx=0x7f5cf4011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:24.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5cf97fa700 1 -- 172.21.15.3:0/3293564360 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf400e040 con 0x7f5cfc1553d0 2023-03-03T06:26:24.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5cf97fa700 1 -- 172.21.15.3:0/3293564360 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf400f040 con 0x7f5cfc1553d0 2023-03-03T06:26:24.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5cf97fa700 1 -- 172.21.15.3:0/3293564360 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf400fdb0 con 0x7f5cfc1553d0 2023-03-03T06:26:24.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.771+0000 7f5d016c4700 1 -- 172.21.15.3:0/3293564360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cfc15d660 con 0x7f5cfc1553d0 2023-03-03T06:26:24.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.775+0000 7f5d016c4700 1 -- 172.21.15.3:0/3293564360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc0cb150 con 0x7f5cfc1553d0 2023-03-03T06:26:24.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.775+0000 7f5d016c4700 1 -- 172.21.15.3:0/3293564360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cfc073e90 con 0x7f5cfc1553d0 2023-03-03T06:26:24.781 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.775+0000 7f5cf97fa700 1 -- 172.21.15.3:0/3293564360 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5cf4022460 con 0x7f5cfc1553d0 2023-03-03T06:26:24.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.779+0000 7f5cf97fa700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5cdc064ee0 0x7f5cdc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:24.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.779+0000 7f5cf97fa700 1 -- 172.21.15.3:0/3293564360 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5cf40829c0 con 0x7f5cfc1553d0 2023-03-03T06:26:24.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.779+0000 7f5cfbfff700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5cdc064ee0 0x7f5cdc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:24.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.779+0000 7f5cfbfff700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5cdc064ee0 0x7f5cdc067390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009bb0 tx=0x7f5cec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.787+0000 7f5cf97fa700 1 -- 172.21.15.3:0/3293564360 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5cf40506e0 con 0x7f5cfc1553d0 2023-03-03T06:26:24.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.967+0000 7f5d016c4700 1 -- 172.21.15.3:0/3293564360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cfc0ccac0 con 0x7f5cfc1553d0 2023-03-03T06:26:24.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.967+0000 7f5cf97fa700 1 -- 172.21.15.3:0/3293564360 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f5cf4053d00 con 0x7f5cfc1553d0 2023-03-03T06:26:24.975 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:24.975 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:53.877945+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:24.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.975+0000 7f5d016c4700 1 -- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5cdc064ee0 msgr2=0x7f5cdc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:24.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.975+0000 7f5d016c4700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5cdc064ee0 0x7f5cdc067390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009bb0 tx=0x7f5cec008040 comp rx=0 tx=0).stop 2023-03-03T06:26:24.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.975+0000 7f5d016c4700 1 -- 172.21.15.3:0/3293564360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cfc1553d0 msgr2=0x7f5cfc15a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:24.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.975+0000 7f5d016c4700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cfc1553d0 0x7f5cfc15a2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0ca3b0 tx=0x7f5cf4011f80 comp rx=0 tx=0).stop 2023-03-03T06:26:24.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.975+0000 7f5d016c4700 1 -- 172.21.15.3:0/3293564360 shutdown_connections 2023-03-03T06:26:24.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.975+0000 7f5d016c4700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5cdc064ee0 0x7f5cdc067390 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.975+0000 7f5d016c4700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5cfc1553d0 0x7f5cfc15a2f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.975+0000 7f5d016c4700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5cfc155a30 0x7f5cfc0ab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.975+0000 7f5d016c4700 1 --2- 172.21.15.3:0/3293564360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce8004870 0x7f5cfc1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:24.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.975+0000 7f5d016c4700 1 -- 172.21.15.3:0/3293564360 >> 172.21.15.3:0/3293564360 conn(0x7f5cfc070a30 msgr2=0x7f5cfc15b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:24.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.975+0000 7f5d016c4700 1 -- 172.21.15.3:0/3293564360 shutdown_connections 2023-03-03T06:26:24.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:24.975+0000 7f5d016c4700 1 -- 172.21.15.3:0/3293564360 wait complete. 2023-03-03T06:26:25.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-03T06:25:11.468534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-03T06:25:12.813283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-03T06:25:16.382800+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-03T06:26:25.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:26:25.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.215+0000 7f1d3ce28700 1 Processor -- start 2023-03-03T06:26:25.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.219+0000 7f1d3ce28700 1 -- start start 2023-03-03T06:26:25.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.219+0000 7f1d3ce28700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d38158400 0x7f1d381587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:25.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.219+0000 7f1d3ce28700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d380cb9a0 0x7f1d380cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.219+0000 7f1d3ce28700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d380cf5f0 con 0x7f1d380709b0 2023-03-03T06:26:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.219+0000 7f1d3ce28700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d380caac0 con 0x7f1d38158400 2023-03-03T06:26:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.219+0000 7f1d3ce28700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3815c230 con 0x7f1d380cb9a0 2023-03-03T06:26:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d377fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d38158400 0x7f1d381587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d377fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d38158400 0x7f1d381587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44742/0 (socket says 172.21.15.3:44742) 2023-03-03T06:26:25.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d377fe700 1 -- 172.21.15.3:0/3106749052 learned_addr learned my addr 172.21.15.3:0/3106749052 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d2ffff700 1 --2- 172.21.15.3:0/3106749052 >> v2:172.21.15.32:3300/0 conn(0x7f1d380cb9a0 0x7f1d380cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:25.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d377fe700 1 -- 172.21.15.3:0/3106749052 >> v2:172.21.15.32:3300/0 conn(0x7f1d380cb9a0 msgr2=0x7f1d380cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d377fe700 1 --2- 172.21.15.3:0/3106749052 >> v2:172.21.15.32:3300/0 conn(0x7f1d380cb9a0 0x7f1d380cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d377fe700 1 -- 172.21.15.3:0/3106749052 >> v1:172.21.15.3:6789/0 conn(0x7f1d380709b0 legacy=0x7f1d3806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d377fe700 1 -- 172.21.15.3:0/3106749052 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d380cd130 con 0x7f1d38158400 2023-03-03T06:26:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d2ffff700 1 --2- 172.21.15.3:0/3106749052 >> v2:172.21.15.32:3300/0 conn(0x7f1d380cb9a0 0x7f1d380cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d36ffd700 1 -- 172.21.15.3:0/3106749052 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180020969 0 0) 0x7f1d380cf5f0 con 0x7f1d380709b0 2023-03-03T06:26:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d377fe700 1 --2- 172.21.15.3:0/3106749052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d38158400 0x7f1d381587d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1d380b0900 tx=0x7f1d2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f71f13e2569204 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:25.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d36ffd700 1 -- 172.21.15.3:0/3106749052 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2800e070 con 0x7f1d38158400 2023-03-03T06:26:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d36ffd700 1 -- 172.21.15.3:0/3106749052 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2800b9c0 con 0x7f1d38158400 2023-03-03T06:26:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d36ffd700 1 -- 172.21.15.3:0/3106749052 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2800bd70 con 0x7f1d38158400 2023-03-03T06:26:25.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d3ce28700 1 -- 172.21.15.3:0/3106749052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d38158400 msgr2=0x7f1d381587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d3ce28700 1 --2- 172.21.15.3:0/3106749052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d38158400 0x7f1d381587d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1d380b0900 tx=0x7f1d2800b330 comp rx=0 tx=0).stop 2023-03-03T06:26:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d3ce28700 1 -- 172.21.15.3:0/3106749052 shutdown_connections 2023-03-03T06:26:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d3ce28700 1 --2- 172.21.15.3:0/3106749052 >> v2:172.21.15.32:3300/0 conn(0x7f1d380cb9a0 0x7f1d380cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d3ce28700 1 --2- 172.21.15.3:0/3106749052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d38158400 0x7f1d381587d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d3ce28700 1 -- 172.21.15.3:0/3106749052 >> 172.21.15.3:0/3106749052 conn(0x7f1d380607c0 msgr2=0x7f1d38060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d3ce28700 1 -- 172.21.15.3:0/3106749052 shutdown_connections 2023-03-03T06:26:25.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.223+0000 7f1d3ce28700 1 -- 172.21.15.3:0/3106749052 wait complete. 2023-03-03T06:26:25.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.227+0000 7f1d3ce28700 1 Processor -- start 2023-03-03T06:26:25.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.227+0000 7f1d3ce28700 1 -- start start 2023-03-03T06:26:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.227+0000 7f1d3ce28700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d380709b0 0x7f1d380af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:25.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.227+0000 7f1d3ce28700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d380cb9a0 0x7f1d380abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:25.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.227+0000 7f1d3ce28700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d38158400 0x7f1d38150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:25.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d3ce28700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380aced0 con 0x7f1d380cb9a0 2023-03-03T06:26:25.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d3ce28700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380ac090 con 0x7f1d380709b0 2023-03-03T06:26:25.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d3ce28700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380ac2c0 con 0x7f1d38158400 2023-03-03T06:26:25.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d2ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d380cb9a0 0x7f1d380abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:25.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d2ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d380cb9a0 0x7f1d380abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44756/0 (socket says 172.21.15.3:44756) 2023-03-03T06:26:25.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d2ffff700 1 -- 172.21.15.3:0/1893956250 learned_addr learned my addr 172.21.15.3:0/1893956250 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:25.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d377fe700 1 --2- 172.21.15.3:0/1893956250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d380709b0 0x7f1d380af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:25.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d2ffff700 1 -- 172.21.15.3:0/1893956250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d38158400 msgr2=0x7f1d38150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:25.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d2ffff700 1 --2- 172.21.15.3:0/1893956250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d38158400 0x7f1d38150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d2ffff700 1 -- 172.21.15.3:0/1893956250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d380709b0 msgr2=0x7f1d380af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:25.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d2ffff700 1 --2- 172.21.15.3:0/1893956250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d380709b0 0x7f1d380af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d2ffff700 1 -- 172.21.15.3:0/1893956250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d28009b30 con 0x7f1d380cb9a0 2023-03-03T06:26:25.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d2ffff700 1 --2- 172.21.15.3:0/1893956250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d380cb9a0 0x7f1d380abb50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000c770 tx=0x7f1d2000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:25.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d34ff9700 1 -- 172.21.15.3:0/1893956250 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20011800 con 0x7f1d380cb9a0 2023-03-03T06:26:25.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d34ff9700 1 -- 172.21.15.3:0/1893956250 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d20011960 con 0x7f1d380cb9a0 2023-03-03T06:26:25.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d3ce28700 1 -- 172.21.15.3:0/1893956250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3815c4f0 con 0x7f1d380cb9a0 2023-03-03T06:26:25.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d34ff9700 1 -- 172.21.15.3:0/1893956250 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20011d10 con 0x7f1d380cb9a0 2023-03-03T06:26:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d3ce28700 1 -- 172.21.15.3:0/1893956250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d380cbf60 con 0x7f1d380cb9a0 2023-03-03T06:26:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.231+0000 7f1d3ce28700 1 -- 172.21.15.3:0/1893956250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3815c7d0 con 0x7f1d380cb9a0 2023-03-03T06:26:25.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.235+0000 7f1d34ff9700 1 -- 172.21.15.3:0/1893956250 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1d20021070 con 0x7f1d380cb9a0 2023-03-03T06:26:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.235+0000 7f1d34ff9700 1 --2- 172.21.15.3:0/1893956250 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d18064ee0 0x7f1d18067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.235+0000 7f1d34ff9700 1 -- 172.21.15.3:0/1893956250 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1d200821d0 con 0x7f1d380cb9a0 2023-03-03T06:26:25.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.239+0000 7f1d377fe700 1 --2- 172.21.15.3:0/1893956250 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d18064ee0 0x7f1d18067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:25.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.239+0000 7f1d377fe700 1 --2- 172.21.15.3:0/1893956250 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d18064ee0 0x7f1d18067390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009f90 tx=0x7f1d28009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:25.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.239+0000 7f1d34ff9700 1 -- 172.21.15.3:0/1893956250 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1d2004ff50 con 0x7f1d380cb9a0 2023-03-03T06:26:25.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.415+0000 7f1d3ce28700 1 -- 172.21.15.3:0/1893956250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d38073e90 con 0x7f1d380cb9a0 2023-03-03T06:26:25.421 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.415+0000 7f1d34ff9700 1 -- 172.21.15.3:0/1893956250 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d20053570 con 0x7f1d380cb9a0 2023-03-03T06:26:25.421 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:26:25.427 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.423+0000 7f1d3ce28700 1 -- 172.21.15.3:0/1893956250 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d18064ee0 msgr2=0x7f1d18067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:25.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.423+0000 7f1d3ce28700 1 --2- 172.21.15.3:0/1893956250 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d18064ee0 0x7f1d18067390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009f90 tx=0x7f1d28009000 comp rx=0 tx=0).stop 2023-03-03T06:26:25.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.423+0000 7f1d3ce28700 1 -- 172.21.15.3:0/1893956250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d380cb9a0 msgr2=0x7f1d380abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:25.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.423+0000 7f1d3ce28700 1 --2- 172.21.15.3:0/1893956250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d380cb9a0 0x7f1d380abb50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000c770 tx=0x7f1d2000cb30 comp rx=0 tx=0).stop 2023-03-03T06:26:25.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.423+0000 7f1d3ce28700 1 -- 172.21.15.3:0/1893956250 shutdown_connections 2023-03-03T06:26:25.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.423+0000 7f1d3ce28700 1 --2- 172.21.15.3:0/1893956250 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1d18064ee0 0x7f1d18067390 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.428 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.423+0000 7f1d3ce28700 1 --2- 172.21.15.3:0/1893956250 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1d38158400 0x7f1d38150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.423+0000 7f1d3ce28700 1 --2- 172.21.15.3:0/1893956250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d380cb9a0 0x7f1d380abb50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.423+0000 7f1d3ce28700 1 --2- 172.21.15.3:0/1893956250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d380709b0 0x7f1d380af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.423+0000 7f1d3ce28700 1 -- 172.21.15.3:0/1893956250 >> 172.21.15.3:0/1893956250 conn(0x7f1d380607c0 msgr2=0x7f1d38067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:25.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.423+0000 7f1d3ce28700 1 -- 172.21.15.3:0/1893956250 shutdown_connections 2023-03-03T06:26:25.429 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.423+0000 7f1d3ce28700 1 -- 172.21.15.3:0/1893956250 wait complete. 2023-03-03T06:26:25.461 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:26:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.663+0000 7f5f430e7700 1 Processor -- start 2023-03-03T06:26:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.667+0000 7f5f430e7700 1 -- start start 2023-03-03T06:26:25.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.667+0000 7f5f430e7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f3c06a780 0x7f5f3c0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:25.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.667+0000 7f5f430e7700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3c15c230 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f430e7700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3c158b10 con 0x7f5f3c0709b0 2023-03-03T06:26:25.674 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f430e7700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3c0caac0 con 0x7f5f3c06a780 2023-03-03T06:26:25.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f418e4700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5f3c0cb9a0 0x7f5f3c158400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53666/0 (socket says 172.21.15.3:53666) 2023-03-03T06:26:25.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f418e4700 1 -- 172.21.15.3:0/2552808014 learned_addr learned my addr 172.21.15.3:0/2552808014 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:25.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f420e5700 1 --2- 172.21.15.3:0/2552808014 >> v2:172.21.15.32:3300/0 conn(0x7f5f3c06a780 0x7f5f3c0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:25.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f410e3700 1 -- 172.21.15.3:0/2552808014 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152210678 0 0) 0x7f5f3c15c230 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f410e3700 1 -- 172.21.15.3:0/2552808014 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f280035d0 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f420e5700 1 -- 172.21.15.3:0/2552808014 >> v1:172.21.15.32:6789/0 conn(0x7f5f3c0709b0 legacy=0x7f5f3c163560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:25.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f420e5700 1 -- 172.21.15.3:0/2552808014 >> v1:172.21.15.3:6789/0 conn(0x7f5f3c0cb9a0 legacy=0x7f5f3c158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:25.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f420e5700 1 -- 172.21.15.3:0/2552808014 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c0cd130 con 0x7f5f3c06a780 2023-03-03T06:26:25.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f420e5700 1 --2- 172.21.15.3:0/2552808014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f3c06a780 0x7f5f3c0cace0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009850 tx=0x7f5f38009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a83dc7943eff268b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:25.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f410e3700 1 -- 172.21.15.3:0/2552808014 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38011830 con 0x7f5f3c06a780 2023-03-03T06:26:25.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f410e3700 1 -- 172.21.15.3:0/2552808014 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f38011990 con 0x7f5f3c06a780 2023-03-03T06:26:25.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.671+0000 7f5f410e3700 1 -- 172.21.15.3:0/2552808014 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38011d40 con 0x7f5f3c06a780 2023-03-03T06:26:25.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.675+0000 7f5f430e7700 1 -- 172.21.15.3:0/2552808014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f3c06a780 msgr2=0x7f5f3c0cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:25.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.675+0000 7f5f430e7700 1 --2- 172.21.15.3:0/2552808014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f3c06a780 0x7f5f3c0cace0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009850 tx=0x7f5f38009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:25.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.675+0000 7f5f430e7700 1 -- 172.21.15.3:0/2552808014 shutdown_connections 2023-03-03T06:26:25.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.675+0000 7f5f430e7700 1 --2- 172.21.15.3:0/2552808014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f3c06a780 0x7f5f3c0cace0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.675+0000 7f5f430e7700 1 -- 172.21.15.3:0/2552808014 >> 172.21.15.3:0/2552808014 conn(0x7f5f3c0607c0 msgr2=0x7f5f3c060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:25.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.675+0000 7f5f430e7700 1 -- 172.21.15.3:0/2552808014 shutdown_connections 2023-03-03T06:26:25.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.675+0000 7f5f430e7700 1 -- 172.21.15.3:0/2552808014 wait complete. 2023-03-03T06:26:25.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.679+0000 7f5f430e7700 1 Processor -- start 2023-03-03T06:26:25.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.679+0000 7f5f430e7700 1 -- start start 2023-03-03T06:26:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.679+0000 7f5f430e7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f3c06a780 0x7f5f3c0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.679+0000 7f5f430e7700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f3c0709b0 0x7f5f3c0ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.679+0000 7f5f430e7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3c0cb9a0 0x7f5f3c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.679+0000 7f5f430e7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c0c7880 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.679+0000 7f5f430e7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c0c69b0 con 0x7f5f3c06a780 2023-03-03T06:26:25.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.679+0000 7f5f430e7700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c0c7ca0 con 0x7f5f3c0709b0 2023-03-03T06:26:25.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.679+0000 7f5f428e6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3c0cb9a0 0x7f5f3c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.679+0000 7f5f428e6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3c0cb9a0 0x7f5f3c152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44762/0 (socket says 172.21.15.3:44762) 2023-03-03T06:26:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.679+0000 7f5f428e6700 1 -- 172.21.15.3:0/3010743298 learned_addr learned my addr 172.21.15.3:0/3010743298 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.679+0000 7f5f420e5700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f3c06a780 0x7f5f3c0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.679+0000 7f5f418e4700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f3c0709b0 0x7f5f3c0ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.683+0000 7f5f428e6700 1 -- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f3c0709b0 msgr2=0x7f5f3c0ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:25.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.683+0000 7f5f428e6700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f3c0709b0 0x7f5f3c0ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.683+0000 7f5f428e6700 1 -- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f3c06a780 msgr2=0x7f5f3c0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:25.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.683+0000 7f5f428e6700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f3c06a780 0x7f5f3c0c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.683+0000 7f5f428e6700 1 -- 172.21.15.3:0/3010743298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380094b0 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.683+0000 7f5f428e6700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3c0cb9a0 0x7f5f3c152880 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400cc80 tx=0x7f5f3400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.683+0000 7f5f32ffd700 1 -- 172.21.15.3:0/3010743298 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34003c20 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.683+0000 7f5f32ffd700 1 -- 172.21.15.3:0/3010743298 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f34003d80 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.683+0000 7f5f32ffd700 1 -- 172.21.15.3:0/3010743298 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400ab50 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.683+0000 7f5f430e7700 1 -- 172.21.15.3:0/3010743298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3c15c4f0 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.683+0000 7f5f430e7700 1 -- 172.21.15.3:0/3010743298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c0cbf60 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.683+0000 7f5f430e7700 1 -- 172.21.15.3:0/3010743298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f3c15c7d0 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.687+0000 7f5f32ffd700 1 -- 172.21.15.3:0/3010743298 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5f34010050 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.687+0000 7f5f32ffd700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5f20064ee0 0x7f5f20067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:25.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.687+0000 7f5f32ffd700 1 -- 172.21.15.3:0/3010743298 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5f34082180 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.691+0000 7f5f420e5700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5f20064ee0 0x7f5f20067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:25.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.691+0000 7f5f420e5700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5f20064ee0 0x7f5f20067390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5f380090b0 tx=0x7f5f38008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:25.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.695+0000 7f5f32ffd700 1 -- 172.21.15.3:0/3010743298 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5f3404fea0 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.881 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.875+0000 7f5f430e7700 1 -- 172.21.15.3:0/3010743298 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5f3c0adeb0 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.882 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.879+0000 7f5f32ffd700 1 -- 172.21.15.3:0/3010743298 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5f340534c0 con 0x7f5f3c0cb9a0 2023-03-03T06:26:25.882 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:26:25.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.879+0000 7f5f430e7700 1 -- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5f20064ee0 msgr2=0x7f5f20067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:25.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.879+0000 7f5f430e7700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5f20064ee0 0x7f5f20067390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5f380090b0 tx=0x7f5f38008030 comp rx=0 tx=0).stop 2023-03-03T06:26:25.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.879+0000 7f5f430e7700 1 -- 172.21.15.3:0/3010743298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3c0cb9a0 msgr2=0x7f5f3c152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:25.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.879+0000 7f5f430e7700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3c0cb9a0 0x7f5f3c152880 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400cc80 tx=0x7f5f3400cf90 comp rx=0 tx=0).stop 2023-03-03T06:26:25.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.879+0000 7f5f430e7700 1 -- 172.21.15.3:0/3010743298 shutdown_connections 2023-03-03T06:26:25.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.879+0000 7f5f430e7700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5f20064ee0 0x7f5f20067390 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.879+0000 7f5f430e7700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f3c0cb9a0 0x7f5f3c152880 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.879+0000 7f5f430e7700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5f3c0709b0 0x7f5f3c0ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.886 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.879+0000 7f5f430e7700 1 --2- 172.21.15.3:0/3010743298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f3c06a780 0x7f5f3c0c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:25.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.879+0000 7f5f430e7700 1 -- 172.21.15.3:0/3010743298 >> 172.21.15.3:0/3010743298 conn(0x7f5f3c0607c0 msgr2=0x7f5f3c06c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:25.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.879+0000 7f5f430e7700 1 -- 172.21.15.3:0/3010743298 shutdown_connections 2023-03-03T06:26:25.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:25.879+0000 7f5f430e7700 1 -- 172.21.15.3:0/3010743298 wait complete. 2023-03-03T06:26:25.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-03T06:26:26.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.119+0000 7fdf7f59e700 1 Processor -- start 2023-03-03T06:26:26.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.123+0000 7fdf7f59e700 1 -- start start 2023-03-03T06:26:26.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.127+0000 7fdf7f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdf8015d6e0 0x7fdf800cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:26.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.127+0000 7fdf7f59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf800cf3b0 con 0x7fdf800cfd80 2023-03-03T06:26:26.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.127+0000 7fdf7f59e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf80156f70 con 0x7fdf80156ba0 2023-03-03T06:26:26.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.127+0000 7fdf7f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf800cd320 con 0x7fdf8015d6e0 2023-03-03T06:26:26.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.127+0000 7fdf7ed9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdf800cfd80 0x7fdf800d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53668/0 (socket says 172.21.15.3:53668) 2023-03-03T06:26:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.127+0000 7fdf7ed9d700 1 -- 172.21.15.3:0/3724573488 learned_addr learned my addr 172.21.15.3:0/3724573488 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.127+0000 7fdf7dd9b700 1 --2- 172.21.15.3:0/3724573488 >> v2:172.21.15.32:3300/0 conn(0x7fdf8015d6e0 0x7fdf800cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.127+0000 7fdf7d59a700 1 -- 172.21.15.3:0/3724573488 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768121404 0 0) 0x7fdf800cf3b0 con 0x7fdf800cfd80 2023-03-03T06:26:26.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.127+0000 7fdf7d59a700 1 -- 172.21.15.3:0/3724573488 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf680035d0 con 0x7fdf800cfd80 2023-03-03T06:26:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.127+0000 7fdf7dd9b700 1 -- 172.21.15.3:0/3724573488 >> v1:172.21.15.32:6789/0 conn(0x7fdf80156ba0 legacy=0x7fdf80071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.127+0000 7fdf7dd9b700 1 -- 172.21.15.3:0/3724573488 >> v1:172.21.15.3:6789/0 conn(0x7fdf800cfd80 legacy=0x7fdf800d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.127+0000 7fdf7dd9b700 1 -- 172.21.15.3:0/3724573488 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf800cd600 con 0x7fdf8015d6e0 2023-03-03T06:26:26.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.131+0000 7fdf7dd9b700 1 --2- 172.21.15.3:0/3724573488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8015d6e0 0x7fdf800cf840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009850 tx=0x7fdf74009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c76c5fac1a67378 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:26.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.131+0000 7fdf7d59a700 1 -- 172.21.15.3:0/3724573488 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf74011830 con 0x7fdf8015d6e0 2023-03-03T06:26:26.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.131+0000 7fdf7d59a700 1 -- 172.21.15.3:0/3724573488 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf74011990 con 0x7fdf8015d6e0 2023-03-03T06:26:26.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.131+0000 7fdf7d59a700 1 -- 172.21.15.3:0/3724573488 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf74011d00 con 0x7fdf8015d6e0 2023-03-03T06:26:26.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.131+0000 7fdf7f59e700 1 -- 172.21.15.3:0/3724573488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8015d6e0 msgr2=0x7fdf800cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:26.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.131+0000 7fdf7f59e700 1 --2- 172.21.15.3:0/3724573488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8015d6e0 0x7fdf800cf840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009850 tx=0x7fdf74009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:26.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.135+0000 7fdf7f59e700 1 -- 172.21.15.3:0/3724573488 shutdown_connections 2023-03-03T06:26:26.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.135+0000 7fdf7f59e700 1 --2- 172.21.15.3:0/3724573488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf8015d6e0 0x7fdf800cf840 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.135+0000 7fdf7f59e700 1 -- 172.21.15.3:0/3724573488 >> 172.21.15.3:0/3724573488 conn(0x7fdf8006e5c0 msgr2=0x7fdf8006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:26.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.135+0000 7fdf7f59e700 1 -- 172.21.15.3:0/3724573488 shutdown_connections 2023-03-03T06:26:26.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.135+0000 7fdf7f59e700 1 -- 172.21.15.3:0/3724573488 wait complete. 2023-03-03T06:26:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.139+0000 7fdf7f59e700 1 Processor -- start 2023-03-03T06:26:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.139+0000 7fdf7f59e700 1 -- start start 2023-03-03T06:26:26.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.139+0000 7fdf7f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf800cfd80 0x7fdf800b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:26.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.139+0000 7fdf7f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf80156ba0 0x7fdf8014dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:26.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.139+0000 7fdf7f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf8015d6e0 0x7fdf80157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:26.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.139+0000 7fdf7f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf800b73a0 con 0x7fdf8015d6e0 2023-03-03T06:26:26.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.139+0000 7fdf7f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80157130 con 0x7fdf800cfd80 2023-03-03T06:26:26.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.139+0000 7fdf7f59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80157380 con 0x7fdf80156ba0 2023-03-03T06:26:26.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf8015d6e0 0x7fdf80157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:26.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7ed9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf8015d6e0 0x7fdf80157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44764/0 (socket says 172.21.15.3:44764) 2023-03-03T06:26:26.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7ed9d700 1 -- 172.21.15.3:0/750832376 learned_addr learned my addr 172.21.15.3:0/750832376 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:26.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7e59c700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf800cfd80 0x7fdf800b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:26.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7dd9b700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf80156ba0 0x7fdf8014dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:26.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7ed9d700 1 -- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf80156ba0 msgr2=0x7fdf8014dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:26.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7ed9d700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf80156ba0 0x7fdf8014dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7ed9d700 1 -- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf800cfd80 msgr2=0x7fdf800b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:26.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7ed9d700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf800cfd80 0x7fdf800b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7ed9d700 1 -- 172.21.15.3:0/750832376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf740094b0 con 0x7fdf8015d6e0 2023-03-03T06:26:26.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7ed9d700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf8015d6e0 0x7fdf80157940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c4f0 tx=0x7fdf7800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:26.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf6effd700 1 -- 172.21.15.3:0/750832376 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78005d90 con 0x7fdf8015d6e0 2023-03-03T06:26:26.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf6effd700 1 -- 172.21.15.3:0/750832376 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf780070d0 con 0x7fdf8015d6e0 2023-03-03T06:26:26.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf6effd700 1 -- 172.21.15.3:0/750832376 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78007440 con 0x7fdf8015d6e0 2023-03-03T06:26:26.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7f59e700 1 -- 172.21.15.3:0/750832376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf800cef40 con 0x7fdf8015d6e0 2023-03-03T06:26:26.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7f59e700 1 -- 172.21.15.3:0/750832376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf800cd8e0 con 0x7fdf8015d6e0 2023-03-03T06:26:26.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.143+0000 7fdf7f59e700 1 -- 172.21.15.3:0/750832376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf800cf0d0 con 0x7fdf8015d6e0 2023-03-03T06:26:26.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.147+0000 7fdf6effd700 1 -- 172.21.15.3:0/750832376 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fdf78019030 con 0x7fdf8015d6e0 2023-03-03T06:26:26.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.147+0000 7fdf6effd700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdf60064ee0 0x7fdf60067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:26.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.147+0000 7fdf6effd700 1 -- 172.21.15.3:0/750832376 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdf780815b0 con 0x7fdf8015d6e0 2023-03-03T06:26:26.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.151+0000 7fdf7e59c700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdf60064ee0 0x7fdf60067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:26.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.151+0000 7fdf7e59c700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdf60064ee0 0x7fdf60067390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf800b87a0 tx=0x7fdf70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:26.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.151+0000 7fdf6effd700 1 -- 172.21.15.3:0/750832376 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fdf7804a400 con 0x7fdf8015d6e0 2023-03-03T06:26:26.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.327+0000 7fdf7f59e700 1 -- 172.21.15.3:0/750832376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf80073e90 con 0x7fdf8015d6e0 2023-03-03T06:26:26.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.327+0000 7fdf6effd700 1 -- 172.21.15.3:0/750832376 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf7804f3e0 con 0x7fdf8015d6e0 2023-03-03T06:26:26.331 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:26.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.331+0000 7fdf7f59e700 1 -- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdf60064ee0 msgr2=0x7fdf60067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:26.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.331+0000 7fdf7f59e700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdf60064ee0 0x7fdf60067390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf800b87a0 tx=0x7fdf70009040 comp rx=0 tx=0).stop 2023-03-03T06:26:26.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.331+0000 7fdf7f59e700 1 -- 172.21.15.3:0/750832376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf8015d6e0 msgr2=0x7fdf80157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:26.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.331+0000 7fdf7f59e700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf8015d6e0 0x7fdf80157940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c4f0 tx=0x7fdf7800c8b0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.331+0000 7fdf7f59e700 1 -- 172.21.15.3:0/750832376 shutdown_connections 2023-03-03T06:26:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.331+0000 7fdf7f59e700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdf60064ee0 0x7fdf60067390 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.331+0000 7fdf7f59e700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf8015d6e0 0x7fdf80157940 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.331+0000 7fdf7f59e700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdf80156ba0 0x7fdf8014dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.331+0000 7fdf7f59e700 1 --2- 172.21.15.3:0/750832376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdf800cfd80 0x7fdf800b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.331+0000 7fdf7f59e700 1 -- 172.21.15.3:0/750832376 >> 172.21.15.3:0/750832376 conn(0x7fdf8006e5c0 msgr2=0x7fdf800b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.331+0000 7fdf7f59e700 1 -- 172.21.15.3:0/750832376 shutdown_connections 2023-03-03T06:26:26.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.331+0000 7fdf7f59e700 1 -- 172.21.15.3:0/750832376 wait complete. 2023-03-03T06:26:26.368 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-03T06:26:26.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.567+0000 7f4d9d830700 1 Processor -- start 2023-03-03T06:26:26.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.567+0000 7f4d9d830700 1 -- start start 2023-03-03T06:26:26.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d9d830700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d9815a4e0 0x7f4d980b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:26.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d9d830700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d980b43d0 0x7f4d9815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:26.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d9d830700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d9815af90 0x7f4d9815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:26.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d9d830700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d980b4850 con 0x7f4d9815a4e0 2023-03-03T06:26:26.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d9d830700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d980ced10 con 0x7f4d980b43d0 2023-03-03T06:26:26.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d9d830700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9815b7c0 con 0x7f4d9815af90 2023-03-03T06:26:26.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d96ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d9815a4e0 0x7f4d980b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:26.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d96ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4d9815a4e0 0x7f4d980b3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44770/0 (socket says 172.21.15.3:44770) 2023-03-03T06:26:26.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d96ffd700 1 -- 172.21.15.3:0/1824468576 learned_addr learned my addr 172.21.15.3:0/1824468576 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:26.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d8ffff700 1 --2- 172.21.15.3:0/1824468576 >> v2:172.21.15.32:3300/0 conn(0x7f4d980b43d0 0x7f4d9815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:26.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d97fff700 1 --2- 172.21.15.3:0/1824468576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d9815af90 0x7f4d9815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:26.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d96ffd700 1 -- 172.21.15.3:0/1824468576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d9815af90 msgr2=0x7f4d9815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:26.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d96ffd700 1 --2- 172.21.15.3:0/1824468576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d9815af90 0x7f4d9815dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d96ffd700 1 -- 172.21.15.3:0/1824468576 >> v2:172.21.15.32:3300/0 conn(0x7f4d980b43d0 msgr2=0x7f4d9815aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:26.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d96ffd700 1 --2- 172.21.15.3:0/1824468576 >> v2:172.21.15.32:3300/0 conn(0x7f4d980b43d0 0x7f4d9815aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d96ffd700 1 -- 172.21.15.3:0/1824468576 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d98070250 con 0x7f4d9815a4e0 2023-03-03T06:26:26.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.571+0000 7f4d8ffff700 1 --2- 172.21.15.3:0/1824468576 >> v2:172.21.15.32:3300/0 conn(0x7f4d980b43d0 0x7f4d9815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:26.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.575+0000 7f4d96ffd700 1 --2- 172.21.15.3:0/1824468576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d9815a4e0 0x7f4d980b3e90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4d9808b800 tx=0x7f4d88009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2d4b61d0acaa9d server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:26.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.575+0000 7f4d967fc700 1 -- 172.21.15.3:0/1824468576 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8800b8a0 con 0x7f4d9815a4e0 2023-03-03T06:26:26.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.575+0000 7f4d967fc700 1 -- 172.21.15.3:0/1824468576 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8800ba00 con 0x7f4d9815a4e0 2023-03-03T06:26:26.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.575+0000 7f4d967fc700 1 -- 172.21.15.3:0/1824468576 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8800bd90 con 0x7f4d9815a4e0 2023-03-03T06:26:26.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.575+0000 7f4d9d830700 1 -- 172.21.15.3:0/1824468576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d9815a4e0 msgr2=0x7f4d980b3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:26.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.575+0000 7f4d9d830700 1 --2- 172.21.15.3:0/1824468576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d9815a4e0 0x7f4d980b3e90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4d9808b800 tx=0x7f4d88009bf0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.575+0000 7f4d9d830700 1 -- 172.21.15.3:0/1824468576 shutdown_connections 2023-03-03T06:26:26.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.575+0000 7f4d9d830700 1 --2- 172.21.15.3:0/1824468576 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d9815af90 0x7f4d9815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.575+0000 7f4d9d830700 1 --2- 172.21.15.3:0/1824468576 >> v2:172.21.15.32:3300/0 conn(0x7f4d980b43d0 0x7f4d9815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.575+0000 7f4d9d830700 1 --2- 172.21.15.3:0/1824468576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d9815a4e0 0x7f4d980b3e90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.575+0000 7f4d9d830700 1 -- 172.21.15.3:0/1824468576 >> 172.21.15.3:0/1824468576 conn(0x7f4d9806cc10 msgr2=0x7f4d980cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:26.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.575+0000 7f4d9d830700 1 -- 172.21.15.3:0/1824468576 shutdown_connections 2023-03-03T06:26:26.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.575+0000 7f4d9d830700 1 -- 172.21.15.3:0/1824468576 wait complete. 2023-03-03T06:26:26.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d9d830700 1 Processor -- start 2023-03-03T06:26:26.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d9d830700 1 -- start start 2023-03-03T06:26:26.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d9d830700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d980b43d0 0x7f4d9815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:26.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d9d830700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d9815a4e0 0x7f4d98143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:26.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d9d830700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d9815af90 0x7f4d9815c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:26.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d9d830700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d980cc370 con 0x7f4d9815af90 2023-03-03T06:26:26.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d9d830700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9814db40 con 0x7f4d980b43d0 2023-03-03T06:26:26.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d9d830700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9814de20 con 0x7f4d9815a4e0 2023-03-03T06:26:26.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d97fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d9815af90 0x7f4d9815c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:26.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d97fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d9815af90 0x7f4d9815c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44782/0 (socket says 172.21.15.3:44782) 2023-03-03T06:26:26.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d97fff700 1 -- 172.21.15.3:0/4013890970 learned_addr learned my addr 172.21.15.3:0/4013890970 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:26.585 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d96ffd700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d980b43d0 0x7f4d9815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d8ffff700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d9815a4e0 0x7f4d98143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d97fff700 1 -- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d9815a4e0 msgr2=0x7f4d98143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d97fff700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d9815a4e0 0x7f4d98143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d97fff700 1 -- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d980b43d0 msgr2=0x7f4d9815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d97fff700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d980b43d0 0x7f4d9815b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d97fff700 1 -- 172.21.15.3:0/4013890970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d88009570 con 0x7f4d9815af90 2023-03-03T06:26:26.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.579+0000 7f4d97fff700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d9815af90 0x7f4d9815c0b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4d90007e60 tx=0x7f4d9000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:26.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.583+0000 7f4d94ff9700 1 -- 172.21.15.3:0/4013890970 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9000ede0 con 0x7f4d9815af90 2023-03-03T06:26:26.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.583+0000 7f4d94ff9700 1 -- 172.21.15.3:0/4013890970 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d90014420 con 0x7f4d9815af90 2023-03-03T06:26:26.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.583+0000 7f4d9d830700 1 -- 172.21.15.3:0/4013890970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d98070810 con 0x7f4d9815af90 2023-03-03T06:26:26.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.583+0000 7f4d94ff9700 1 -- 172.21.15.3:0/4013890970 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d900147d0 con 0x7f4d9815af90 2023-03-03T06:26:26.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.583+0000 7f4d9d830700 1 -- 172.21.15.3:0/4013890970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d98070530 con 0x7f4d9815af90 2023-03-03T06:26:26.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.583+0000 7f4d9d830700 1 -- 172.21.15.3:0/4013890970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d98070af0 con 0x7f4d9815af90 2023-03-03T06:26:26.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.587+0000 7f4d94ff9700 1 -- 172.21.15.3:0/4013890970 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4d90012050 con 0x7f4d9815af90 2023-03-03T06:26:26.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.587+0000 7f4d94ff9700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d78064ee0 0x7f4d78067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:26.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.587+0000 7f4d94ff9700 1 -- 172.21.15.3:0/4013890970 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4d90082850 con 0x7f4d9815af90 2023-03-03T06:26:26.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.591+0000 7f4d96ffd700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d78064ee0 0x7f4d78067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:26.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.591+0000 7f4d96ffd700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d78064ee0 0x7f4d78067390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4d88000c00 tx=0x7f4d88008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.595+0000 7f4d94ff9700 1 -- 172.21.15.3:0/4013890970 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4d90050570 con 0x7f4d9815af90 2023-03-03T06:26:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.771+0000 7f4d9d830700 1 -- 172.21.15.3:0/4013890970 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d9804e180 con 0x7f4d9815af90 2023-03-03T06:26:26.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.775+0000 7f4d94ff9700 1 -- 172.21.15.3:0/4013890970 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d90053b90 con 0x7f4d9815af90 2023-03-03T06:26:26.778 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-03-03T06:26:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.779+0000 7f4d9d830700 1 -- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d78064ee0 msgr2=0x7f4d78067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.779+0000 7f4d9d830700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d78064ee0 0x7f4d78067390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4d88000c00 tx=0x7f4d88008030 comp rx=0 tx=0).stop 2023-03-03T06:26:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.779+0000 7f4d9d830700 1 -- 172.21.15.3:0/4013890970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d9815af90 msgr2=0x7f4d9815c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.779+0000 7f4d9d830700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d9815af90 0x7f4d9815c0b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4d90007e60 tx=0x7f4d9000e3c0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.779+0000 7f4d9d830700 1 -- 172.21.15.3:0/4013890970 shutdown_connections 2023-03-03T06:26:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.779+0000 7f4d9d830700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4d78064ee0 0x7f4d78067390 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.779+0000 7f4d9d830700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d9815af90 0x7f4d9815c0b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.779+0000 7f4d9d830700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4d9815a4e0 0x7f4d98143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.779+0000 7f4d9d830700 1 --2- 172.21.15.3:0/4013890970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d980b43d0 0x7f4d9815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:26.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.779+0000 7f4d9d830700 1 -- 172.21.15.3:0/4013890970 >> 172.21.15.3:0/4013890970 conn(0x7f4d9806cc10 msgr2=0x7f4d9806b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:26.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.779+0000 7f4d9d830700 1 -- 172.21.15.3:0/4013890970 shutdown_connections 2023-03-03T06:26:26.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:26.779+0000 7f4d9d830700 1 -- 172.21.15.3:0/4013890970 wait complete. 2023-03-03T06:26:26.816 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-03T06:26:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.023+0000 7f5ce4923700 1 Processor -- start 2023-03-03T06:26:27.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.023+0000 7f5ce4923700 1 -- start start 2023-03-03T06:26:27.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.027+0000 7f5ce4923700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce015d6e0 0x7f5ce00cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:27.033 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.027+0000 7f5ce4923700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ce00cfd80 0x7f5ce00d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:27.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.027+0000 7f5ce4923700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce00cf3b0 con 0x7f5ce0156ba0 2023-03-03T06:26:27.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.031+0000 7f5ce4923700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce0156f70 con 0x7f5ce00cfd80 2023-03-03T06:26:27.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.031+0000 7f5ce4923700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce00cd320 con 0x7f5ce015d6e0 2023-03-03T06:26:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.031+0000 7f5cdf7fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ce0156ba0 0x7f5ce0071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:46080/0 (socket says 172.21.15.3:46080) 2023-03-03T06:26:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.031+0000 7f5cdf7fe700 1 -- 172.21.15.3:0/1824565491 learned_addr learned my addr 172.21.15.3:0/1824565491 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.031+0000 7f5cdffff700 1 --2- 172.21.15.3:0/1824565491 >> v2:172.21.15.32:3300/0 conn(0x7f5ce00cfd80 0x7f5ce00d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:27.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.031+0000 7f5cd6ffd700 1 --2- 172.21.15.3:0/1824565491 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce015d6e0 0x7f5ce00cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:27.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.031+0000 7f5cdeffd700 1 -- 172.21.15.3:0/1824565491 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120226755 0 0) 0x7f5ce00cf3b0 con 0x7f5ce0156ba0 2023-03-03T06:26:27.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.031+0000 7f5cdeffd700 1 -- 172.21.15.3:0/1824565491 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ccc0035d0 con 0x7f5ce0156ba0 2023-03-03T06:26:27.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.031+0000 7f5cdffff700 1 -- 172.21.15.3:0/1824565491 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce015d6e0 msgr2=0x7f5ce00cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:27.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.031+0000 7f5cdffff700 1 --2- 172.21.15.3:0/1824565491 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce015d6e0 0x7f5ce00cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.031+0000 7f5cdffff700 1 -- 172.21.15.3:0/1824565491 >> v1:172.21.15.32:6789/0 conn(0x7f5ce0156ba0 legacy=0x7f5ce0071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:27.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.031+0000 7f5cdffff700 1 -- 172.21.15.3:0/1824565491 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce00cd600 con 0x7f5ce00cfd80 2023-03-03T06:26:27.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.031+0000 7f5cdffff700 1 --2- 172.21.15.3:0/1824565491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce00cfd80 0x7f5ce00d0b30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009d90 tx=0x7f5cd800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da2eeb73141aa11 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:27.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.035+0000 7f5cdeffd700 1 -- 172.21.15.3:0/1824565491 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd800e070 con 0x7f5ce00cfd80 2023-03-03T06:26:27.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.035+0000 7f5cdeffd700 1 -- 172.21.15.3:0/1824565491 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd800bb20 con 0x7f5ce00cfd80 2023-03-03T06:26:27.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.035+0000 7f5cdeffd700 1 -- 172.21.15.3:0/1824565491 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8012430 con 0x7f5ce00cfd80 2023-03-03T06:26:27.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.035+0000 7f5ce4923700 1 -- 172.21.15.3:0/1824565491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce00cfd80 msgr2=0x7f5ce00d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:27.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.035+0000 7f5ce4923700 1 --2- 172.21.15.3:0/1824565491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce00cfd80 0x7f5ce00d0b30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009d90 tx=0x7f5cd800b330 comp rx=0 tx=0).stop 2023-03-03T06:26:27.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.035+0000 7f5ce4923700 1 -- 172.21.15.3:0/1824565491 shutdown_connections 2023-03-03T06:26:27.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.035+0000 7f5ce4923700 1 --2- 172.21.15.3:0/1824565491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce00cfd80 0x7f5ce00d0b30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.035+0000 7f5ce4923700 1 --2- 172.21.15.3:0/1824565491 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce015d6e0 0x7f5ce00cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.035+0000 7f5ce4923700 1 -- 172.21.15.3:0/1824565491 >> 172.21.15.3:0/1824565491 conn(0x7f5ce006e5c0 msgr2=0x7f5ce006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:27.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.035+0000 7f5ce4923700 1 -- 172.21.15.3:0/1824565491 shutdown_connections 2023-03-03T06:26:27.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.035+0000 7f5ce4923700 1 -- 172.21.15.3:0/1824565491 wait complete. 2023-03-03T06:26:27.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.039+0000 7f5ce4923700 1 Processor -- start 2023-03-03T06:26:27.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.039+0000 7f5ce4923700 1 -- start start 2023-03-03T06:26:27.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5ce4923700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce00cfd80 0x7f5ce01f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:27.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5ce4923700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce0156ba0 0x7f5ce00cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:27.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5ce4923700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce015d6e0 0x7f5ce00cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:27.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5ce4923700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce00cb8f0 con 0x7f5ce015d6e0 2023-03-03T06:26:27.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5ce4923700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce00c8640 con 0x7f5ce00cfd80 2023-03-03T06:26:27.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5ce4923700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce00c8920 con 0x7f5ce0156ba0 2023-03-03T06:26:27.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce015d6e0 0x7f5ce00cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:27.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce015d6e0 0x7f5ce00cc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44794/0 (socket says 172.21.15.3:44794) 2023-03-03T06:26:27.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdffff700 1 -- 172.21.15.3:0/342227442 learned_addr learned my addr 172.21.15.3:0/342227442 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:27.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cd6ffd700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce0156ba0 0x7f5ce00cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:27.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdf7fe700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce00cfd80 0x7f5ce01f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:27.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdffff700 1 -- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce0156ba0 msgr2=0x7f5ce00cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:27.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdffff700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce0156ba0 0x7f5ce00cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdffff700 1 -- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce00cfd80 msgr2=0x7f5ce01f14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:27.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdffff700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce00cfd80 0x7f5ce01f14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdffff700 1 -- 172.21.15.3:0/342227442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd8009a70 con 0x7f5ce015d6e0 2023-03-03T06:26:27.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdf7fe700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce00cfd80 0x7f5ce01f14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:27.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdffff700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce015d6e0 0x7f5ce00cc050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8014040 tx=0x7f5cd8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:27.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdcff9700 1 -- 172.21.15.3:0/342227442 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8012430 con 0x7f5ce015d6e0 2023-03-03T06:26:27.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdcff9700 1 -- 172.21.15.3:0/342227442 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd8020070 con 0x7f5ce015d6e0 2023-03-03T06:26:27.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.043+0000 7f5cdcff9700 1 -- 172.21.15.3:0/342227442 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8004280 con 0x7f5ce015d6e0 2023-03-03T06:26:27.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.047+0000 7f5ce4923700 1 -- 172.21.15.3:0/342227442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce00cef40 con 0x7f5ce015d6e0 2023-03-03T06:26:27.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.047+0000 7f5ce4923700 1 -- 172.21.15.3:0/342227442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce00cd8e0 con 0x7f5ce015d6e0 2023-03-03T06:26:27.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.047+0000 7f5ce4923700 1 -- 172.21.15.3:0/342227442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce00cf0d0 con 0x7f5ce015d6e0 2023-03-03T06:26:27.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.051+0000 7f5cdcff9700 1 -- 172.21.15.3:0/342227442 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5cd802e070 con 0x7f5ce015d6e0 2023-03-03T06:26:27.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.051+0000 7f5cdcff9700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5cc0064ee0 0x7f5cc0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:27.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.051+0000 7f5cdcff9700 1 -- 172.21.15.3:0/342227442 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5cd8022070 con 0x7f5ce015d6e0 2023-03-03T06:26:27.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.051+0000 7f5cdf7fe700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5cc0064ee0 0x7f5cc0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:27.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.055+0000 7f5cdf7fe700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5cc0064ee0 0x7f5cc0067390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00011e0 tx=0x7f5cd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:27.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.059+0000 7f5cdcff9700 1 -- 172.21.15.3:0/342227442 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5cd8051bf0 con 0x7f5ce015d6e0 2023-03-03T06:26:27.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.235+0000 7f5ce4923700 1 -- 172.21.15.3:0/342227442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5ce0073e90 con 0x7f5ce015d6e0 2023-03-03T06:26:27.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.235+0000 7f5cdcff9700 1 -- 172.21.15.3:0/342227442 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5cd802c0a0 con 0x7f5ce015d6e0 2023-03-03T06:26:27.240 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:27.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.239+0000 7f5ce4923700 1 -- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5cc0064ee0 msgr2=0x7f5cc0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:27.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.239+0000 7f5ce4923700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5cc0064ee0 0x7f5cc0067390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00011e0 tx=0x7f5cd0009040 comp rx=0 tx=0).stop 2023-03-03T06:26:27.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.239+0000 7f5ce4923700 1 -- 172.21.15.3:0/342227442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce015d6e0 msgr2=0x7f5ce00cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:27.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.239+0000 7f5ce4923700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce015d6e0 0x7f5ce00cc050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8014040 tx=0x7f5cd8004af0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.239+0000 7f5ce4923700 1 -- 172.21.15.3:0/342227442 shutdown_connections 2023-03-03T06:26:27.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.239+0000 7f5ce4923700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5cc0064ee0 0x7f5cc0067390 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.239+0000 7f5ce4923700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ce015d6e0 0x7f5ce00cc050 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.239+0000 7f5ce4923700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5ce0156ba0 0x7f5ce00cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.239+0000 7f5ce4923700 1 --2- 172.21.15.3:0/342227442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ce00cfd80 0x7f5ce01f14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.239+0000 7f5ce4923700 1 -- 172.21.15.3:0/342227442 >> 172.21.15.3:0/342227442 conn(0x7f5ce006e5c0 msgr2=0x7f5ce015f380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:27.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.243+0000 7f5ce4923700 1 -- 172.21.15.3:0/342227442 shutdown_connections 2023-03-03T06:26:27.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.243+0000 7f5ce4923700 1 -- 172.21.15.3:0/342227442 wait complete. 2023-03-03T06:26:27.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-03T06:26:27.366 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:26:27.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:26:27.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.475+0000 7f505a89f700 1 Processor -- start 2023-03-03T06:26:27.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.475+0000 7f505a89f700 1 -- start start 2023-03-03T06:26:27.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.479+0000 7f505a89f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f50541608d0 0x7f50540cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:27.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.479+0000 7f505a89f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50540cfb60 0x7f50540d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:27.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.479+0000 7f505a89f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5054160440 con 0x7f5054153670 2023-03-03T06:26:27.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.479+0000 7f505a89f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50540cffa0 con 0x7f50541608d0 2023-03-03T06:26:27.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.479+0000 7f505a89f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50540cd3e0 con 0x7f50540cfb60 2023-03-03T06:26:27.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505909c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f50541608d0 0x7f50540cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:27.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505a09e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50540cfb60 0x7f50540d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:27.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505989d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5054153670 0x7f5054071350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:46090/0 (socket says 172.21.15.3:46090) 2023-03-03T06:26:27.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505989d700 1 -- 172.21.15.3:0/1678555083 learned_addr learned my addr 172.21.15.3:0/1678555083 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:27.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505a09e700 1 -- 172.21.15.3:0/1678555083 >> v2:172.21.15.3:3300/0 conn(0x7f50541608d0 msgr2=0x7f50540cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:27.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505a09e700 1 --2- 172.21.15.3:0/1678555083 >> v2:172.21.15.3:3300/0 conn(0x7f50541608d0 0x7f50540cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505a09e700 1 -- 172.21.15.3:0/1678555083 >> v1:172.21.15.32:6789/0 conn(0x7f5054153670 legacy=0x7f5054071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:27.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505a09e700 1 -- 172.21.15.3:0/1678555083 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50540cd6c0 con 0x7f50540cfb60 2023-03-03T06:26:27.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505a09e700 1 --2- 172.21.15.3:0/1678555083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50540cfb60 0x7f50540d0b10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f504c009850 tx=0x7f504c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b274ce86db758c44 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:27.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505889b700 1 -- 172.21.15.3:0/1678555083 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c011830 con 0x7f50540cfb60 2023-03-03T06:26:27.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505889b700 1 -- 172.21.15.3:0/1678555083 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c011990 con 0x7f50540cfb60 2023-03-03T06:26:27.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505889b700 1 -- 172.21.15.3:0/1678555083 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c011d20 con 0x7f50540cfb60 2023-03-03T06:26:27.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505a89f700 1 -- 172.21.15.3:0/1678555083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50540cfb60 msgr2=0x7f50540d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:27.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505a89f700 1 --2- 172.21.15.3:0/1678555083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50540cfb60 0x7f50540d0b10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f504c009850 tx=0x7f504c009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:27.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505a89f700 1 -- 172.21.15.3:0/1678555083 shutdown_connections 2023-03-03T06:26:27.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505a89f700 1 --2- 172.21.15.3:0/1678555083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50540cfb60 0x7f50540d0b10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505a89f700 1 --2- 172.21.15.3:0/1678555083 >> v2:172.21.15.3:3300/0 conn(0x7f50541608d0 0x7f50540cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.483+0000 7f505a89f700 1 -- 172.21.15.3:0/1678555083 >> 172.21.15.3:0/1678555083 conn(0x7f505406e5c0 msgr2=0x7f505406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:27.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.487+0000 7f505a89f700 1 -- 172.21.15.3:0/1678555083 shutdown_connections 2023-03-03T06:26:27.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.487+0000 7f505a89f700 1 -- 172.21.15.3:0/1678555083 wait complete. 2023-03-03T06:26:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.487+0000 7f505a89f700 1 Processor -- start 2023-03-03T06:26:27.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.487+0000 7f505a89f700 1 -- start start 2023-03-03T06:26:27.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a89f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50540cfb60 0x7f505414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:27.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a89f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5054153670 0x7f50540b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:27.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a89f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50541608d0 0x7f50541507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:27.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a89f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50540b4100 con 0x7f50541608d0 2023-03-03T06:26:27.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a89f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50540b31e0 con 0x7f5054153670 2023-03-03T06:26:27.495 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a89f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50540b3480 con 0x7f50540cfb60 2023-03-03T06:26:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a09e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50541608d0 0x7f50541507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a09e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50541608d0 0x7f50541507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44798/0 (socket says 172.21.15.3:44798) 2023-03-03T06:26:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a09e700 1 -- 172.21.15.3:0/3343031656 learned_addr learned my addr 172.21.15.3:0/3343031656 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505909c700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5054153670 0x7f50540b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:27.496 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505989d700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50540cfb60 0x7f505414d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:27.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a09e700 1 -- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50540cfb60 msgr2=0x7f505414d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:27.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a09e700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50540cfb60 0x7f505414d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a09e700 1 -- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5054153670 msgr2=0x7f50540b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:27.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a09e700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5054153670 0x7f50540b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.497 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a09e700 1 -- 172.21.15.3:0/3343031656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c0094b0 con 0x7f50541608d0 2023-03-03T06:26:27.498 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a09e700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50541608d0 0x7f50541507e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f504c0090b0 tx=0x7f504c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:27.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f504a7fc700 1 -- 172.21.15.3:0/3343031656 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c0036c0 con 0x7f50541608d0 2023-03-03T06:26:27.499 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f504a7fc700 1 -- 172.21.15.3:0/3343031656 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c01f070 con 0x7f50541608d0 2023-03-03T06:26:27.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f504a7fc700 1 -- 172.21.15.3:0/3343031656 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c003a40 con 0x7f50541608d0 2023-03-03T06:26:27.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a89f700 1 -- 172.21.15.3:0/3343031656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50540cd9a0 con 0x7f50541608d0 2023-03-03T06:26:27.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.491+0000 7f505a89f700 1 -- 172.21.15.3:0/3343031656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50541600d0 con 0x7f50541608d0 2023-03-03T06:26:27.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.495+0000 7f505a89f700 1 -- 172.21.15.3:0/3343031656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50540cdb30 con 0x7f50541608d0 2023-03-03T06:26:27.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.495+0000 7f504a7fc700 1 -- 172.21.15.3:0/3343031656 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f504c00f050 con 0x7f50541608d0 2023-03-03T06:26:27.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.499+0000 7f504a7fc700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5038064ee0 0x7f5038067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:27.503 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.499+0000 7f504a7fc700 1 -- 172.21.15.3:0/3343031656 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f504c021070 con 0x7f50541608d0 2023-03-03T06:26:27.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.499+0000 7f505989d700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5038064ee0 0x7f5038067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:27.505 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.503+0000 7f505989d700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5038064ee0 0x7f5038067390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5050005cb0 tx=0x7f5050009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:27.508 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.503+0000 7f504a7fc700 1 -- 172.21.15.3:0/3343031656 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f504c051940 con 0x7f50541608d0 2023-03-03T06:26:27.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.691+0000 7f505a89f700 1 -- 172.21.15.3:0/3343031656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f50540b4240 con 0x7f50541608d0 2023-03-03T06:26:27.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.695+0000 7f504a7fc700 1 -- 172.21.15.3:0/3343031656 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f504c015080 con 0x7f50541608d0 2023-03-03T06:26:27.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.695+0000 7f505a89f700 1 -- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5038064ee0 msgr2=0x7f5038067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:27.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.695+0000 7f505a89f700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5038064ee0 0x7f5038067390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5050005cb0 tx=0x7f5050009040 comp rx=0 tx=0).stop 2023-03-03T06:26:27.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.695+0000 7f505a89f700 1 -- 172.21.15.3:0/3343031656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50541608d0 msgr2=0x7f50541507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:27.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.695+0000 7f505a89f700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50541608d0 0x7f50541507e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f504c0090b0 tx=0x7f504c004770 comp rx=0 tx=0).stop 2023-03-03T06:26:27.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.695+0000 7f505a89f700 1 -- 172.21.15.3:0/3343031656 shutdown_connections 2023-03-03T06:26:27.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.695+0000 7f505a89f700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5038064ee0 0x7f5038067390 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.695+0000 7f505a89f700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50541608d0 0x7f50541507e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.695+0000 7f505a89f700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5054153670 0x7f50540b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.695+0000 7f505a89f700 1 --2- 172.21.15.3:0/3343031656 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f50540cfb60 0x7f505414d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:27.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.695+0000 7f505a89f700 1 -- 172.21.15.3:0/3343031656 >> 172.21.15.3:0/3343031656 conn(0x7f505406e5c0 msgr2=0x7f505415dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:27.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.699+0000 7f505a89f700 1 -- 172.21.15.3:0/3343031656 shutdown_connections 2023-03-03T06:26:27.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:27.699+0000 7f505a89f700 1 -- 172.21.15.3:0/3343031656 wait complete. 2023-03-03T06:26:27.704 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2023-03-03T06:26:27.731 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-03-03T06:26:27.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:27.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-03-03T06:26:27.850 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2023-03-03T06:26:27.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:27.850 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-03-03T06:26:27.878 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:26:27.879 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:27.902 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:26:27.902 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:27.964 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-03-03T06:26:27.965 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:26:28.029 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:28.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.235+0000 7f5ba2234700 1 Processor -- start 2023-03-03T06:26:28.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.239+0000 7f5ba2234700 1 -- start start 2023-03-03T06:26:28.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.239+0000 7f5ba2234700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b9c070820 0x7f5b9c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:28.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.239+0000 7f5ba2234700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b9c156d60 con 0x7f5b9c15d360 2023-03-03T06:26:28.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.239+0000 7f5ba2234700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b9c0cd9e0 con 0x7f5b9c0cccb0 2023-03-03T06:26:28.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.239+0000 7f5ba2234700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9c0cd800 con 0x7f5b9c070820 2023-03-03T06:26:28.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.243+0000 7f5ba1232700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5b9c15d360 0x7f5b9c15da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53684/0 (socket says 172.21.15.3:53684) 2023-03-03T06:26:28.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.243+0000 7f5ba1232700 1 -- 172.21.15.3:0/625458907 learned_addr learned my addr 172.21.15.3:0/625458907 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:28.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.243+0000 7f5ba0a31700 1 --2- 172.21.15.3:0/625458907 >> v2:172.21.15.32:3300/0 conn(0x7f5b9c070820 0x7f5b9c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:28.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.243+0000 7f5b93fff700 1 -- 172.21.15.3:0/625458907 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794850902 0 0) 0x7f5b98001490 con 0x7f5b9c15d360 2023-03-03T06:26:28.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.243+0000 7f5b93fff700 1 -- 172.21.15.3:0/625458907 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b880035d0 con 0x7f5b9c15d360 2023-03-03T06:26:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.243+0000 7f5b93fff700 1 -- 172.21.15.3:0/625458907 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48649705 0 0) 0x7f5b9c0cd9e0 con 0x7f5b9c0cccb0 2023-03-03T06:26:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.243+0000 7f5b93fff700 1 -- 172.21.15.3:0/625458907 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b98001490 con 0x7f5b9c0cccb0 2023-03-03T06:26:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.243+0000 7f5ba0a31700 1 -- 172.21.15.3:0/625458907 >> v1:172.21.15.32:6789/0 conn(0x7f5b9c0cccb0 legacy=0x7f5b9c0cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.243+0000 7f5ba0a31700 1 -- 172.21.15.3:0/625458907 >> v1:172.21.15.3:6789/0 conn(0x7f5b9c15d360 legacy=0x7f5b9c15da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.243+0000 7f5b93fff700 1 -- 172.21.15.3:0/625458907 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1360008209 0 0) 0x7f5b880035d0 con 0x7f5b9c15d360 2023-03-03T06:26:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.243+0000 7f5ba0a31700 1 -- 172.21.15.3:0/625458907 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c0c96c0 con 0x7f5b9c070820 2023-03-03T06:26:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.243+0000 7f5ba0a31700 1 --2- 172.21.15.3:0/625458907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c070820 0x7f5b9c070cc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009850 tx=0x7f5b8c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d9868006b47a7d6 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:28.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.247+0000 7f5b93fff700 1 -- 172.21.15.3:0/625458907 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98001b80 con 0x7f5b9c070820 2023-03-03T06:26:28.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.247+0000 7f5b93fff700 1 -- 172.21.15.3:0/625458907 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8c004030 con 0x7f5b9c070820 2023-03-03T06:26:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.247+0000 7f5b93fff700 1 -- 172.21.15.3:0/625458907 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0043c0 con 0x7f5b9c070820 2023-03-03T06:26:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.247+0000 7f5ba2234700 1 -- 172.21.15.3:0/625458907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c070820 msgr2=0x7f5b9c070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.247+0000 7f5ba2234700 1 --2- 172.21.15.3:0/625458907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c070820 0x7f5b9c070cc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009850 tx=0x7f5b8c009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.247+0000 7f5ba2234700 1 -- 172.21.15.3:0/625458907 shutdown_connections 2023-03-03T06:26:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.247+0000 7f5ba2234700 1 --2- 172.21.15.3:0/625458907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c070820 0x7f5b9c070cc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.247+0000 7f5ba2234700 1 -- 172.21.15.3:0/625458907 >> 172.21.15.3:0/625458907 conn(0x7f5b9c0cf370 msgr2=0x7f5b9c0cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.247+0000 7f5ba2234700 1 -- 172.21.15.3:0/625458907 shutdown_connections 2023-03-03T06:26:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.247+0000 7f5ba2234700 1 -- 172.21.15.3:0/625458907 wait complete. 2023-03-03T06:26:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.251+0000 7f5ba2234700 1 Processor -- start 2023-03-03T06:26:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.251+0000 7f5ba2234700 1 -- start start 2023-03-03T06:26:28.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.251+0000 7f5ba2234700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b9c070820 0x7f5b9c0b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:28.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.251+0000 7f5ba2234700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5b9c0cccb0 0x7f5b9c153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:28.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.251+0000 7f5ba2234700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c15d360 0x7f5b9c155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:28.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.251+0000 7f5ba2234700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c153c70 con 0x7f5b9c070820 2023-03-03T06:26:28.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.251+0000 7f5ba2234700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c155480 con 0x7f5b9c15d360 2023-03-03T06:26:28.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.251+0000 7f5ba2234700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c154870 con 0x7f5b9c0cccb0 2023-03-03T06:26:28.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba1232700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b9c070820 0x7f5b9c0b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:28.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba0a31700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5b9c0cccb0 0x7f5b9c153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:28.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba1232700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b9c070820 0x7f5b9c0b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44814/0 (socket says 172.21.15.3:44814) 2023-03-03T06:26:28.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba0a31700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5b9c0cccb0 0x7f5b9c153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:43272/0 (socket says 172.21.15.3:43272) 2023-03-03T06:26:28.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba1232700 1 -- 172.21.15.3:0/928309256 learned_addr learned my addr 172.21.15.3:0/928309256 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:28.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba1a33700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c15d360 0x7f5b9c155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:28.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba1232700 1 -- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5b9c0cccb0 msgr2=0x7f5b9c153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:28.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba1232700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5b9c0cccb0 0x7f5b9c153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba1232700 1 -- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c15d360 msgr2=0x7f5b9c155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:28.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba1232700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c15d360 0x7f5b9c155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba1232700 1 -- 172.21.15.3:0/928309256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c0094b0 con 0x7f5b9c070820 2023-03-03T06:26:28.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba1a33700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c15d360 0x7f5b9c155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:28.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba1232700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b9c070820 0x7f5b9c0b0040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5b98001b60 tx=0x7f5b9800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:28.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5b91ffb700 1 -- 172.21.15.3:0/928309256 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9800acd0 con 0x7f5b9c070820 2023-03-03T06:26:28.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5b91ffb700 1 -- 172.21.15.3:0/928309256 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9800ae70 con 0x7f5b9c070820 2023-03-03T06:26:28.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba2234700 1 -- 172.21.15.3:0/928309256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9c0c9c80 con 0x7f5b9c070820 2023-03-03T06:26:28.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5b91ffb700 1 -- 172.21.15.3:0/928309256 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b980072f0 con 0x7f5b9c070820 2023-03-03T06:26:28.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.255+0000 7f5ba2234700 1 -- 172.21.15.3:0/928309256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9c0c99a0 con 0x7f5b9c070820 2023-03-03T06:26:28.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.259+0000 7f5ba2234700 1 -- 172.21.15.3:0/928309256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9c0c9f60 con 0x7f5b9c070820 2023-03-03T06:26:28.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.259+0000 7f5b91ffb700 1 -- 172.21.15.3:0/928309256 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5b98010070 con 0x7f5b9c070820 2023-03-03T06:26:28.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.263+0000 7f5b91ffb700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5b80064f10 0x7f5b800673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.263+0000 7f5b91ffb700 1 -- 172.21.15.3:0/928309256 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5b98081d00 con 0x7f5b9c070820 2023-03-03T06:26:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.263+0000 7f5ba0a31700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5b80064f10 0x7f5b800673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.267+0000 7f5ba0a31700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5b80064f10 0x7f5b800673c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0090b0 tx=0x7f5b8c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:28.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.267+0000 7f5b91ffb700 1 -- 172.21.15.3:0/928309256 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5b9804fa80 con 0x7f5b9c070820 2023-03-03T06:26:28.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.447+0000 7f5ba2234700 1 -- 172.21.15.3:0/928309256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b9c156d60 con 0x7f5b9c070820 2023-03-03T06:26:28.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.447+0000 7f5b91ffb700 1 -- 172.21.15.3:0/928309256 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f5b9c156d60 con 0x7f5b9c070820 2023-03-03T06:26:28.453 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:28.453 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:53.877945+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.455+0000 7f5ba2234700 1 -- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5b80064f10 msgr2=0x7f5b800673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:28.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.455+0000 7f5ba2234700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5b80064f10 0x7f5b800673c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0090b0 tx=0x7f5b8c008030 comp rx=0 tx=0).stop 2023-03-03T06:26:28.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.455+0000 7f5ba2234700 1 -- 172.21.15.3:0/928309256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b9c070820 msgr2=0x7f5b9c0b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:28.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.455+0000 7f5ba2234700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b9c070820 0x7f5b9c0b0040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5b98001b60 tx=0x7f5b9800a340 comp rx=0 tx=0).stop 2023-03-03T06:26:28.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.455+0000 7f5ba2234700 1 -- 172.21.15.3:0/928309256 shutdown_connections 2023-03-03T06:26:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.455+0000 7f5ba2234700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5b80064f10 0x7f5b800673c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.455+0000 7f5ba2234700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b9c15d360 0x7f5b9c155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.455+0000 7f5ba2234700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5b9c0cccb0 0x7f5b9c153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.455+0000 7f5ba2234700 1 --2- 172.21.15.3:0/928309256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b9c070820 0x7f5b9c0b0040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.455+0000 7f5ba2234700 1 -- 172.21.15.3:0/928309256 >> 172.21.15.3:0/928309256 conn(0x7f5b9c0cf370 msgr2=0x7f5b9c0cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.455+0000 7f5ba2234700 1 -- 172.21.15.3:0/928309256 shutdown_connections 2023-03-03T06:26:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.455+0000 7f5ba2234700 1 -- 172.21.15.3:0/928309256 wait complete. 2023-03-03T06:26:28.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-03T06:25:11.468534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-03T06:25:12.813283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-03T06:25:16.382800+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-03T06:26:28.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:26:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.699+0000 7efd85728700 1 Processor -- start 2023-03-03T06:26:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.699+0000 7efd85728700 1 -- start start 2023-03-03T06:26:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.699+0000 7efd85728700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd800710c0 0x7efd800cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.699+0000 7efd85728700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd800cb9a0 0x7efd800ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.699+0000 7efd85728700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8015c230 con 0x7efd8006a780 2023-03-03T06:26:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.699+0000 7efd85728700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd800caac0 con 0x7efd800710c0 2023-03-03T06:26:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.699+0000 7efd85728700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd800cd130 con 0x7efd800cb9a0 2023-03-03T06:26:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.699+0000 7efd777fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd800710c0 0x7efd800cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.699+0000 7efd84f27700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efd800cb9a0 0x7efd800ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.699+0000 7efd777fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd800710c0 0x7efd800cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44820/0 (socket says 172.21.15.3:44820) 2023-03-03T06:26:28.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.699+0000 7efd777fe700 1 -- 172.21.15.3:0/4246613644 learned_addr learned my addr 172.21.15.3:0/4246613644 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd777fe700 1 -- 172.21.15.3:0/4246613644 >> v2:172.21.15.32:3300/0 conn(0x7efd800cb9a0 msgr2=0x7efd800ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd777fe700 1 --2- 172.21.15.3:0/4246613644 >> v2:172.21.15.32:3300/0 conn(0x7efd800cb9a0 0x7efd800ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd777fe700 1 -- 172.21.15.3:0/4246613644 >> v1:172.21.15.32:6789/0 conn(0x7efd8006a780 legacy=0x7efd800709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd777fe700 1 -- 172.21.15.3:0/4246613644 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd800cbf60 con 0x7efd800710c0 2023-03-03T06:26:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd777fe700 1 --2- 172.21.15.3:0/4246613644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd800710c0 0x7efd800cace0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efd800c5830 tx=0x7efd6800aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a812c5c230a3305 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd7f7fe700 1 -- 172.21.15.3:0/4246613644 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68004230 con 0x7efd800710c0 2023-03-03T06:26:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd7f7fe700 1 -- 172.21.15.3:0/4246613644 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd68004390 con 0x7efd800710c0 2023-03-03T06:26:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd7f7fe700 1 -- 172.21.15.3:0/4246613644 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd680036c0 con 0x7efd800710c0 2023-03-03T06:26:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd85728700 1 -- 172.21.15.3:0/4246613644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd800710c0 msgr2=0x7efd800cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd85728700 1 --2- 172.21.15.3:0/4246613644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd800710c0 0x7efd800cace0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efd800c5830 tx=0x7efd6800aca0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd85728700 1 -- 172.21.15.3:0/4246613644 shutdown_connections 2023-03-03T06:26:28.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd85728700 1 --2- 172.21.15.3:0/4246613644 >> v2:172.21.15.32:3300/0 conn(0x7efd800cb9a0 0x7efd800ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd85728700 1 --2- 172.21.15.3:0/4246613644 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd800710c0 0x7efd800cace0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd85728700 1 -- 172.21.15.3:0/4246613644 >> 172.21.15.3:0/4246613644 conn(0x7efd800607c0 msgr2=0x7efd80060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:28.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd85728700 1 -- 172.21.15.3:0/4246613644 shutdown_connections 2023-03-03T06:26:28.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.703+0000 7efd85728700 1 -- 172.21.15.3:0/4246613644 wait complete. 2023-03-03T06:26:28.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.707+0000 7efd85728700 1 Processor -- start 2023-03-03T06:26:28.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.707+0000 7efd85728700 1 -- start start 2023-03-03T06:26:28.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.707+0000 7efd85728700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd8006a780 0x7efd8014db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:28.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.707+0000 7efd85728700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd800710c0 0x7efd800b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:28.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.707+0000 7efd85728700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd800cb9a0 0x7efd800c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:28.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.707+0000 7efd85728700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800b1940 con 0x7efd8006a780 2023-03-03T06:26:28.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.707+0000 7efd85728700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800a8a40 con 0x7efd800cb9a0 2023-03-03T06:26:28.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.707+0000 7efd85728700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd800a81a0 con 0x7efd800710c0 2023-03-03T06:26:28.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.707+0000 7efd7ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd8006a780 0x7efd8014db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:28.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd7ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd8006a780 0x7efd8014db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44824/0 (socket says 172.21.15.3:44824) 2023-03-03T06:26:28.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd7ffff700 1 -- 172.21.15.3:0/3629296421 learned_addr learned my addr 172.21.15.3:0/3629296421 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:28.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd84f27700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd800cb9a0 0x7efd800c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:28.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd777fe700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd800710c0 0x7efd800b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:28.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd7ffff700 1 -- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd800710c0 msgr2=0x7efd800b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:28.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd7ffff700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd800710c0 0x7efd800b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd7ffff700 1 -- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd800cb9a0 msgr2=0x7efd800c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:28.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd7ffff700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd800cb9a0 0x7efd800c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd7ffff700 1 -- 172.21.15.3:0/3629296421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6800a620 con 0x7efd8006a780 2023-03-03T06:26:28.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd7ffff700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd8006a780 0x7efd8014db10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c500 tx=0x7efd7000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:28.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd7d7fa700 1 -- 172.21.15.3:0/3629296421 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd700052d0 con 0x7efd8006a780 2023-03-03T06:26:28.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd7d7fa700 1 -- 172.21.15.3:0/3629296421 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd70005430 con 0x7efd8006a780 2023-03-03T06:26:28.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd7d7fa700 1 -- 172.21.15.3:0/3629296421 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd700055a0 con 0x7efd8006a780 2023-03-03T06:26:28.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd85728700 1 -- 172.21.15.3:0/3629296421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8015c7d0 con 0x7efd8006a780 2023-03-03T06:26:28.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.711+0000 7efd85728700 1 -- 172.21.15.3:0/3629296421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8015c4f0 con 0x7efd8006a780 2023-03-03T06:26:28.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.715+0000 7efd85728700 1 -- 172.21.15.3:0/3629296421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8015cab0 con 0x7efd8006a780 2023-03-03T06:26:28.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.715+0000 7efd7d7fa700 1 -- 172.21.15.3:0/3629296421 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7efd70019030 con 0x7efd8006a780 2023-03-03T06:26:28.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.715+0000 7efd7d7fa700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efd64064ee0 0x7efd64067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:28.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.715+0000 7efd7d7fa700 1 -- 172.21.15.3:0/3629296421 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efd70081510 con 0x7efd8006a780 2023-03-03T06:26:28.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.719+0000 7efd777fe700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efd64064ee0 0x7efd64067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:28.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.719+0000 7efd777fe700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efd64064ee0 0x7efd64067390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efd800c5830 tx=0x7efd68011000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:28.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.723+0000 7efd7d7fa700 1 -- 172.21.15.3:0/3629296421 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7efd7004f230 con 0x7efd8006a780 2023-03-03T06:26:28.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.899+0000 7efd85728700 1 -- 172.21.15.3:0/3629296421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd800b16a0 con 0x7efd8006a780 2023-03-03T06:26:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.899+0000 7efd7d7fa700 1 -- 172.21.15.3:0/3629296421 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd70052850 con 0x7efd8006a780 2023-03-03T06:26:28.904 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:26:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.903+0000 7efd85728700 1 -- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efd64064ee0 msgr2=0x7efd64067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.903+0000 7efd85728700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efd64064ee0 0x7efd64067390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efd800c5830 tx=0x7efd68011000 comp rx=0 tx=0).stop 2023-03-03T06:26:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.903+0000 7efd85728700 1 -- 172.21.15.3:0/3629296421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd8006a780 msgr2=0x7efd8014db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.903+0000 7efd85728700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd8006a780 0x7efd8014db10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c500 tx=0x7efd7000c810 comp rx=0 tx=0).stop 2023-03-03T06:26:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.903+0000 7efd85728700 1 -- 172.21.15.3:0/3629296421 shutdown_connections 2023-03-03T06:26:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.903+0000 7efd85728700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efd64064ee0 0x7efd64067390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.903+0000 7efd85728700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd800cb9a0 0x7efd800c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.903+0000 7efd85728700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efd800710c0 0x7efd800b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.903+0000 7efd85728700 1 --2- 172.21.15.3:0/3629296421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd8006a780 0x7efd8014db10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.903+0000 7efd85728700 1 -- 172.21.15.3:0/3629296421 >> 172.21.15.3:0/3629296421 conn(0x7efd800607c0 msgr2=0x7efd801591c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.903+0000 7efd85728700 1 -- 172.21.15.3:0/3629296421 shutdown_connections 2023-03-03T06:26:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:28.903+0000 7efd85728700 1 -- 172.21.15.3:0/3629296421 wait complete. 2023-03-03T06:26:28.942 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:26:29.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.151+0000 7f7314fa8700 1 Processor -- start 2023-03-03T06:26:29.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.155+0000 7f7314fa8700 1 -- start start 2023-03-03T06:26:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.155+0000 7f7314fa8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7310156ba0 0x7f7310071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.155+0000 7f7314fa8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73100cf3b0 con 0x7f731015d6e0 2023-03-03T06:26:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.155+0000 7f7314fa8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7310156f70 con 0x7f73100d0b30 2023-03-03T06:26:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.155+0000 7f7314fa8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73100cd320 con 0x7f7310156ba0 2023-03-03T06:26:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.159+0000 7f730effd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f731015d6e0 0x7f73100cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53690/0 (socket says 172.21.15.3:53690) 2023-03-03T06:26:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.159+0000 7f730effd700 1 -- 172.21.15.3:0/621991894 learned_addr learned my addr 172.21.15.3:0/621991894 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.159+0000 7f730f7fe700 1 --2- 172.21.15.3:0/621991894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7310156ba0 0x7f7310071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:29.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.159+0000 7f730e7fc700 1 -- 172.21.15.3:0/621991894 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705671023 0 0) 0x7f73100cf3b0 con 0x7f731015d6e0 2023-03-03T06:26:29.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.159+0000 7f730e7fc700 1 -- 172.21.15.3:0/621991894 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72f80035d0 con 0x7f731015d6e0 2023-03-03T06:26:29.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.159+0000 7f730f7fe700 1 -- 172.21.15.3:0/621991894 >> v1:172.21.15.32:6789/0 conn(0x7f73100d0b30 legacy=0x7f73100b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.159+0000 7f730e7fc700 1 -- 172.21.15.3:0/621991894 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546695744 0 0) 0x7f7310156f70 con 0x7f73100d0b30 2023-03-03T06:26:29.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.159+0000 7f730f7fe700 1 -- 172.21.15.3:0/621991894 >> v1:172.21.15.3:6789/0 conn(0x7f731015d6e0 legacy=0x7f73100cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.159+0000 7f730f7fe700 1 -- 172.21.15.3:0/621991894 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73100cd600 con 0x7f7310156ba0 2023-03-03T06:26:29.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.159+0000 7f730f7fe700 1 --2- 172.21.15.3:0/621991894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7310156ba0 0x7f7310071280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7300009880 tx=0x7f7300009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e136acf014508686 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:29.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.159+0000 7f730e7fc700 1 -- 172.21.15.3:0/621991894 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300011830 con 0x7f7310156ba0 2023-03-03T06:26:29.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.159+0000 7f730e7fc700 1 -- 172.21.15.3:0/621991894 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7300011990 con 0x7f7310156ba0 2023-03-03T06:26:29.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.159+0000 7f730e7fc700 1 -- 172.21.15.3:0/621991894 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300011d20 con 0x7f7310156ba0 2023-03-03T06:26:29.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.163+0000 7f7314fa8700 1 -- 172.21.15.3:0/621991894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7310156ba0 msgr2=0x7f7310071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.163+0000 7f7314fa8700 1 --2- 172.21.15.3:0/621991894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7310156ba0 0x7f7310071280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7300009880 tx=0x7f7300009b90 comp rx=0 tx=0).stop 2023-03-03T06:26:29.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.163+0000 7f7314fa8700 1 -- 172.21.15.3:0/621991894 shutdown_connections 2023-03-03T06:26:29.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.163+0000 7f7314fa8700 1 --2- 172.21.15.3:0/621991894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7310156ba0 0x7f7310071280 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.163+0000 7f7314fa8700 1 -- 172.21.15.3:0/621991894 >> 172.21.15.3:0/621991894 conn(0x7f731006e5c0 msgr2=0x7f731006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:29.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.163+0000 7f7314fa8700 1 -- 172.21.15.3:0/621991894 shutdown_connections 2023-03-03T06:26:29.167 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.163+0000 7f7314fa8700 1 -- 172.21.15.3:0/621991894 wait complete. 2023-03-03T06:26:29.169 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.163+0000 7f7314fa8700 1 Processor -- start 2023-03-03T06:26:29.170 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f7314fa8700 1 -- start start 2023-03-03T06:26:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f7314fa8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73100d0b30 0x7f7310151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f7314fa8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7310156ba0 0x7f73100cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f7314fa8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f731015d6e0 0x7f73100b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f7314fa8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73100b8510 con 0x7f7310156ba0 2023-03-03T06:26:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f7314fa8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73100cb5b0 con 0x7f73100d0b30 2023-03-03T06:26:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f7314fa8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73100b5d90 con 0x7f731015d6e0 2023-03-03T06:26:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f730effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7310156ba0 0x7f73100cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f730effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7310156ba0 0x7f73100cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44826/0 (socket says 172.21.15.3:44826) 2023-03-03T06:26:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f730effd700 1 -- 172.21.15.3:0/2007761971 learned_addr learned my addr 172.21.15.3:0/2007761971 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f730ffff700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f731015d6e0 0x7f73100b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f730f7fe700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73100d0b30 0x7f7310151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f730effd700 1 -- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f731015d6e0 msgr2=0x7f73100b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f730effd700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f731015d6e0 0x7f73100b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f730effd700 1 -- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73100d0b30 msgr2=0x7f7310151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f730effd700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73100d0b30 0x7f7310151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f730effd700 1 -- 172.21.15.3:0/2007761971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73000094b0 con 0x7f7310156ba0 2023-03-03T06:26:29.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.167+0000 7f730effd700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7310156ba0 0x7f73100cc810 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f730400c500 tx=0x7f730400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:29.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.171+0000 7f72f7fff700 1 -- 172.21.15.3:0/2007761971 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304005d90 con 0x7f7310156ba0 2023-03-03T06:26:29.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.171+0000 7f7314fa8700 1 -- 172.21.15.3:0/2007761971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73100717c0 con 0x7f7310156ba0 2023-03-03T06:26:29.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.171+0000 7f72f7fff700 1 -- 172.21.15.3:0/2007761971 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730400ce70 con 0x7f7310156ba0 2023-03-03T06:26:29.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.171+0000 7f72f7fff700 1 -- 172.21.15.3:0/2007761971 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73040072d0 con 0x7f7310156ba0 2023-03-03T06:26:29.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.171+0000 7f7314fa8700 1 -- 172.21.15.3:0/2007761971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73100cd8e0 con 0x7f7310156ba0 2023-03-03T06:26:29.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.171+0000 7f7314fa8700 1 -- 172.21.15.3:0/2007761971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73100cff50 con 0x7f7310156ba0 2023-03-03T06:26:29.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.171+0000 7f72f7fff700 1 -- 172.21.15.3:0/2007761971 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7304019030 con 0x7f7310156ba0 2023-03-03T06:26:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.175+0000 7f72f7fff700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f72fc064ee0 0x7f72fc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:29.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.175+0000 7f72f7fff700 1 -- 172.21.15.3:0/2007761971 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7304081340 con 0x7f7310156ba0 2023-03-03T06:26:29.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.175+0000 7f730f7fe700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f72fc064ee0 0x7f72fc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:29.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.175+0000 7f730f7fe700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f72fc064ee0 0x7f72fc067390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f73000090b0 tx=0x7f730001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:29.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.179+0000 7f72f7fff700 1 -- 172.21.15.3:0/2007761971 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f730404a400 con 0x7f7310156ba0 2023-03-03T06:26:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.359+0000 7f7314fa8700 1 -- 172.21.15.3:0/2007761971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7310073e90 con 0x7f7310156ba0 2023-03-03T06:26:29.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.363+0000 7f72f7fff700 1 -- 172.21.15.3:0/2007761971 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7304017020 con 0x7f7310156ba0 2023-03-03T06:26:29.366 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:26:29.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.363+0000 7f7314fa8700 1 -- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f72fc064ee0 msgr2=0x7f72fc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.363+0000 7f7314fa8700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f72fc064ee0 0x7f72fc067390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f73000090b0 tx=0x7f730001c040 comp rx=0 tx=0).stop 2023-03-03T06:26:29.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.363+0000 7f7314fa8700 1 -- 172.21.15.3:0/2007761971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7310156ba0 msgr2=0x7f73100cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.363+0000 7f7314fa8700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7310156ba0 0x7f73100cc810 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f730400c500 tx=0x7f730400c810 comp rx=0 tx=0).stop 2023-03-03T06:26:29.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.363+0000 7f7314fa8700 1 -- 172.21.15.3:0/2007761971 shutdown_connections 2023-03-03T06:26:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.363+0000 7f7314fa8700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f72fc064ee0 0x7f72fc067390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.363+0000 7f7314fa8700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f731015d6e0 0x7f73100b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.363+0000 7f7314fa8700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7310156ba0 0x7f73100cc810 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.363+0000 7f7314fa8700 1 --2- 172.21.15.3:0/2007761971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73100d0b30 0x7f7310151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.363+0000 7f7314fa8700 1 -- 172.21.15.3:0/2007761971 >> 172.21.15.3:0/2007761971 conn(0x7f731006e5c0 msgr2=0x7f731015c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.367+0000 7f7314fa8700 1 -- 172.21.15.3:0/2007761971 shutdown_connections 2023-03-03T06:26:29.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.367+0000 7f7314fa8700 1 -- 172.21.15.3:0/2007761971 wait complete. 2023-03-03T06:26:29.401 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-03T06:26:29.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.607+0000 7fe1bb084700 1 Processor -- start 2023-03-03T06:26:29.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.607+0000 7fe1bb084700 1 -- start start 2023-03-03T06:26:29.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.611+0000 7fe1bb084700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b40ca850 0x7fe1b40cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:29.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.611+0000 7fe1bb084700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b40cac20 con 0x7fe1b41608a0 2023-03-03T06:26:29.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.611+0000 7fe1bb084700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b415ca10 con 0x7fe1b40b0860 2023-03-03T06:26:29.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.611+0000 7fe1bb084700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b4160ee0 con 0x7fe1b40ca850 2023-03-03T06:26:29.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.611+0000 7fe1b9881700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe1b41608a0 0x7fe1b406c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53696/0 (socket says 172.21.15.3:53696) 2023-03-03T06:26:29.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.611+0000 7fe1b9881700 1 -- 172.21.15.3:0/3102393011 learned_addr learned my addr 172.21.15.3:0/3102393011 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:29.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.611+0000 7fe1ba082700 1 --2- 172.21.15.3:0/3102393011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b40ca850 0x7fe1b40cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:29.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.615+0000 7fe1b9080700 1 -- 172.21.15.3:0/3102393011 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160204591 0 0) 0x7fe1a4001110 con 0x7fe1b41608a0 2023-03-03T06:26:29.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.615+0000 7fe1b9080700 1 -- 172.21.15.3:0/3102393011 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1a00035d0 con 0x7fe1b41608a0 2023-03-03T06:26:29.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.615+0000 7fe1b9080700 1 -- 172.21.15.3:0/3102393011 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244601905 0 0) 0x7fe1b415ca10 con 0x7fe1b40b0860 2023-03-03T06:26:29.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.615+0000 7fe1b9080700 1 -- 172.21.15.3:0/3102393011 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1a4001110 con 0x7fe1b40b0860 2023-03-03T06:26:29.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.615+0000 7fe1ba082700 1 -- 172.21.15.3:0/3102393011 >> v1:172.21.15.32:6789/0 conn(0x7fe1b40b0860 legacy=0x7fe1b40b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.615+0000 7fe1ba082700 1 -- 172.21.15.3:0/3102393011 >> v1:172.21.15.3:6789/0 conn(0x7fe1b41608a0 legacy=0x7fe1b406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.615+0000 7fe1ba082700 1 -- 172.21.15.3:0/3102393011 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b40cc520 con 0x7fe1b40ca850 2023-03-03T06:26:29.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.615+0000 7fe1ba082700 1 --2- 172.21.15.3:0/3102393011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b40ca850 0x7fe1b40cb6d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009880 tx=0x7fe1b0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd8b185477ab158a server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:29.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.615+0000 7fe1b9080700 1 -- 172.21.15.3:0/3102393011 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0011830 con 0x7fe1b40ca850 2023-03-03T06:26:29.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.615+0000 7fe1b9080700 1 -- 172.21.15.3:0/3102393011 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b0011990 con 0x7fe1b40ca850 2023-03-03T06:26:29.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.615+0000 7fe1b9080700 1 -- 172.21.15.3:0/3102393011 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0011d20 con 0x7fe1b40ca850 2023-03-03T06:26:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.619+0000 7fe1bb084700 1 -- 172.21.15.3:0/3102393011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b40ca850 msgr2=0x7fe1b40cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.619+0000 7fe1bb084700 1 --2- 172.21.15.3:0/3102393011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b40ca850 0x7fe1b40cb6d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009880 tx=0x7fe1b0009b90 comp rx=0 tx=0).stop 2023-03-03T06:26:29.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.619+0000 7fe1bb084700 1 -- 172.21.15.3:0/3102393011 shutdown_connections 2023-03-03T06:26:29.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.619+0000 7fe1bb084700 1 --2- 172.21.15.3:0/3102393011 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b40ca850 0x7fe1b40cb6d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.619+0000 7fe1bb084700 1 -- 172.21.15.3:0/3102393011 >> 172.21.15.3:0/3102393011 conn(0x7fe1b40607c0 msgr2=0x7fe1b4060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.619+0000 7fe1bb084700 1 -- 172.21.15.3:0/3102393011 shutdown_connections 2023-03-03T06:26:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.619+0000 7fe1bb084700 1 -- 172.21.15.3:0/3102393011 wait complete. 2023-03-03T06:26:29.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1bb084700 1 Processor -- start 2023-03-03T06:26:29.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1bb084700 1 -- start start 2023-03-03T06:26:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1bb084700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b40b0860 0x7fe1b4159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1bb084700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b40ca850 0x7fe1b40c2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1bb084700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b41608a0 0x7fe1b415f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1bb084700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b415eb90 con 0x7fe1b40b0860 2023-03-03T06:26:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1bb084700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b415e260 con 0x7fe1b41608a0 2023-03-03T06:26:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1bb084700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b415e540 con 0x7fe1b40ca850 2023-03-03T06:26:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1ba082700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b40b0860 0x7fe1b4159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1ba082700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b40b0860 0x7fe1b4159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44832/0 (socket says 172.21.15.3:44832) 2023-03-03T06:26:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1ba082700 1 -- 172.21.15.3:0/2450415464 learned_addr learned my addr 172.21.15.3:0/2450415464 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1b9881700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b40ca850 0x7fe1b40c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1ba883700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b41608a0 0x7fe1b415f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1ba082700 1 -- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b40ca850 msgr2=0x7fe1b40c2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1ba082700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b40ca850 0x7fe1b40c2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1ba082700 1 -- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b41608a0 msgr2=0x7fe1b415f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1ba082700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b41608a0 0x7fe1b415f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1ba082700 1 -- 172.21.15.3:0/2450415464 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b00094b0 con 0x7fe1b40b0860 2023-03-03T06:26:29.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.623+0000 7fe1ba883700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b41608a0 0x7fe1b415f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:29.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.627+0000 7fe1ba082700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b40b0860 0x7fe1b4159360 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0013040 tx=0x7fe1b0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:29.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.627+0000 7fe1aaffd700 1 -- 172.21.15.3:0/2450415464 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b001a8c0 con 0x7fe1b40b0860 2023-03-03T06:26:29.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.627+0000 7fe1aaffd700 1 -- 172.21.15.3:0/2450415464 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b001e070 con 0x7fe1b40b0860 2023-03-03T06:26:29.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.627+0000 7fe1aaffd700 1 -- 172.21.15.3:0/2450415464 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b001ac40 con 0x7fe1b40b0860 2023-03-03T06:26:29.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.627+0000 7fe1bb084700 1 -- 172.21.15.3:0/2450415464 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b406c7a0 con 0x7fe1b40b0860 2023-03-03T06:26:29.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.627+0000 7fe1bb084700 1 -- 172.21.15.3:0/2450415464 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b40cc800 con 0x7fe1b40b0860 2023-03-03T06:26:29.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.627+0000 7fe1bb084700 1 -- 172.21.15.3:0/2450415464 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b40ce410 con 0x7fe1b40b0860 2023-03-03T06:26:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.631+0000 7fe1aaffd700 1 -- 172.21.15.3:0/2450415464 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe1b000f050 con 0x7fe1b40b0860 2023-03-03T06:26:29.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.631+0000 7fe1aaffd700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe198064ee0 0x7fe198067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:29.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.635+0000 7fe1aaffd700 1 -- 172.21.15.3:0/2450415464 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe1b0083130 con 0x7fe1b40b0860 2023-03-03T06:26:29.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.635+0000 7fe1b9881700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe198064ee0 0x7fe198067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:29.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.635+0000 7fe1b9881700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe198064ee0 0x7fe198067390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe1b415e030 tx=0x7fe1a4007e20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.639+0000 7fe1aaffd700 1 -- 172.21.15.3:0/2450415464 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe1b0050eb0 con 0x7fe1b40b0860 2023-03-03T06:26:29.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.815+0000 7fe1bb084700 1 -- 172.21.15.3:0/2450415464 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1b4073e90 con 0x7fe1b40b0860 2023-03-03T06:26:29.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.815+0000 7fe1aaffd700 1 -- 172.21.15.3:0/2450415464 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1b0018310 con 0x7fe1b40b0860 2023-03-03T06:26:29.822 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:29.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.819+0000 7fe1bb084700 1 -- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe198064ee0 msgr2=0x7fe198067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.819+0000 7fe1bb084700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe198064ee0 0x7fe198067390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe1b415e030 tx=0x7fe1a4007e20 comp rx=0 tx=0).stop 2023-03-03T06:26:29.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.819+0000 7fe1bb084700 1 -- 172.21.15.3:0/2450415464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b40b0860 msgr2=0x7fe1b4159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:29.825 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.819+0000 7fe1bb084700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b40b0860 0x7fe1b4159360 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0013040 tx=0x7fe1b0004770 comp rx=0 tx=0).stop 2023-03-03T06:26:29.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.819+0000 7fe1bb084700 1 -- 172.21.15.3:0/2450415464 shutdown_connections 2023-03-03T06:26:29.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.819+0000 7fe1bb084700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe198064ee0 0x7fe198067390 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.819+0000 7fe1bb084700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b41608a0 0x7fe1b415f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.819+0000 7fe1bb084700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe1b40ca850 0x7fe1b40c2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.819+0000 7fe1bb084700 1 --2- 172.21.15.3:0/2450415464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1b40b0860 0x7fe1b4159360 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:29.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.819+0000 7fe1bb084700 1 -- 172.21.15.3:0/2450415464 >> 172.21.15.3:0/2450415464 conn(0x7fe1b40607c0 msgr2=0x7fe1b415fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:29.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.819+0000 7fe1bb084700 1 -- 172.21.15.3:0/2450415464 shutdown_connections 2023-03-03T06:26:29.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:29.819+0000 7fe1bb084700 1 -- 172.21.15.3:0/2450415464 wait complete. 2023-03-03T06:26:29.858 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-03T06:26:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.071+0000 7f904639a700 1 Processor -- start 2023-03-03T06:26:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.071+0000 7f904639a700 1 -- start start 2023-03-03T06:26:30.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.075+0000 7f904639a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f904015d6e0 0x7f90400cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.075+0000 7f904639a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90400cfd80 0x7f90400d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.075+0000 7f904639a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90400cf3b0 con 0x7f9040156ba0 2023-03-03T06:26:30.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.075+0000 7f904639a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9040156f70 con 0x7f904015d6e0 2023-03-03T06:26:30.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.075+0000 7f904639a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90400cd320 con 0x7f90400cfd80 2023-03-03T06:26:30.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.075+0000 7f9044b97700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f904015d6e0 0x7f90400cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.075+0000 7f9045398700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9040156ba0 0x7f9040071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53698/0 (socket says 172.21.15.3:53698) 2023-03-03T06:26:30.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.075+0000 7f9045398700 1 -- 172.21.15.3:0/2812520776 learned_addr learned my addr 172.21.15.3:0/2812520776 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.075+0000 7f9045b99700 1 --2- 172.21.15.3:0/2812520776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90400cfd80 0x7f90400d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.075+0000 7f9044b97700 1 -- 172.21.15.3:0/2812520776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90400cfd80 msgr2=0x7f90400d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.075+0000 7f9044b97700 1 --2- 172.21.15.3:0/2812520776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90400cfd80 0x7f90400d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f9044b97700 1 -- 172.21.15.3:0/2812520776 >> v1:172.21.15.3:6789/0 conn(0x7f9040156ba0 legacy=0x7f9040071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f9037fff700 1 -- 172.21.15.3:0/2812520776 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635836915 0 0) 0x7f90400cf3b0 con 0x7f9040156ba0 2023-03-03T06:26:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f9044b97700 1 -- 172.21.15.3:0/2812520776 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90400cd600 con 0x7f904015d6e0 2023-03-03T06:26:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f9044b97700 1 --2- 172.21.15.3:0/2812520776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904015d6e0 0x7f90400cf840 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9030009850 tx=0x7f9030009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cecdd072992c3d37 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f9037fff700 1 -- 172.21.15.3:0/2812520776 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030011830 con 0x7f904015d6e0 2023-03-03T06:26:30.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f9037fff700 1 -- 172.21.15.3:0/2812520776 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9030011990 con 0x7f904015d6e0 2023-03-03T06:26:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f9037fff700 1 -- 172.21.15.3:0/2812520776 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030011d20 con 0x7f904015d6e0 2023-03-03T06:26:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f904639a700 1 -- 172.21.15.3:0/2812520776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904015d6e0 msgr2=0x7f90400cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f904639a700 1 --2- 172.21.15.3:0/2812520776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904015d6e0 0x7f90400cf840 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9030009850 tx=0x7f9030009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:30.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f904639a700 1 -- 172.21.15.3:0/2812520776 shutdown_connections 2023-03-03T06:26:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f904639a700 1 --2- 172.21.15.3:0/2812520776 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f90400cfd80 0x7f90400d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f904639a700 1 --2- 172.21.15.3:0/2812520776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904015d6e0 0x7f90400cf840 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f904639a700 1 -- 172.21.15.3:0/2812520776 >> 172.21.15.3:0/2812520776 conn(0x7f904006e5c0 msgr2=0x7f904006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f904639a700 1 -- 172.21.15.3:0/2812520776 shutdown_connections 2023-03-03T06:26:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.079+0000 7f904639a700 1 -- 172.21.15.3:0/2812520776 wait complete. 2023-03-03T06:26:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.083+0000 7f904639a700 1 Processor -- start 2023-03-03T06:26:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.083+0000 7f904639a700 1 -- start start 2023-03-03T06:26:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f904639a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90400cfd80 0x7f90400b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f904639a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9040156ba0 0x7f90400b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f904639a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f904015d6e0 0x7f90400c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f904639a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90400b3e00 con 0x7f90400cfd80 2023-03-03T06:26:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f904639a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9040151980 con 0x7f9040156ba0 2023-03-03T06:26:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f904639a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90400b3560 con 0x7f904015d6e0 2023-03-03T06:26:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f9045398700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90400cfd80 0x7f90400b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f9045398700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90400cfd80 0x7f90400b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44858/0 (socket says 172.21.15.3:44858) 2023-03-03T06:26:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f9045398700 1 -- 172.21.15.3:0/4141026948 learned_addr learned my addr 172.21.15.3:0/4141026948 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f9045b99700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f904015d6e0 0x7f90400c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f9044b97700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9040156ba0 0x7f90400b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f9045398700 1 -- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f904015d6e0 msgr2=0x7f90400c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f9045398700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f904015d6e0 0x7f90400c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f9045398700 1 -- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9040156ba0 msgr2=0x7f90400b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f9045398700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9040156ba0 0x7f90400b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f9045398700 1 -- 172.21.15.3:0/4141026948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90300094b0 con 0x7f90400cfd80 2023-03-03T06:26:30.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f9044b97700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9040156ba0 0x7f90400b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:30.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.087+0000 7f9045398700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90400cfd80 0x7f90400b65e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f903c00c5b0 tx=0x7f903c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.091+0000 7f9035ffb700 1 -- 172.21.15.3:0/4141026948 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c004ca0 con 0x7f90400cfd80 2023-03-03T06:26:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.091+0000 7f9035ffb700 1 -- 172.21.15.3:0/4141026948 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903c004e20 con 0x7f90400cfd80 2023-03-03T06:26:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.091+0000 7f904639a700 1 -- 172.21.15.3:0/4141026948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90400cef40 con 0x7f90400cfd80 2023-03-03T06:26:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.091+0000 7f9035ffb700 1 -- 172.21.15.3:0/4141026948 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c007310 con 0x7f90400cfd80 2023-03-03T06:26:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.091+0000 7f904639a700 1 -- 172.21.15.3:0/4141026948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90400cd8e0 con 0x7f90400cfd80 2023-03-03T06:26:30.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.091+0000 7f904639a700 1 -- 172.21.15.3:0/4141026948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90400cf0d0 con 0x7f90400cfd80 2023-03-03T06:26:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.095+0000 7f9035ffb700 1 -- 172.21.15.3:0/4141026948 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f903c01a020 con 0x7f90400cfd80 2023-03-03T06:26:30.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.095+0000 7f9035ffb700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9024064ee0 0x7f9024067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.095+0000 7f9035ffb700 1 -- 172.21.15.3:0/4141026948 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f903c081e50 con 0x7f90400cfd80 2023-03-03T06:26:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.099+0000 7f9044b97700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9024064ee0 0x7f9024067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.103 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.099+0000 7f9044b97700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9024064ee0 0x7f9024067390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9030000c00 tx=0x7f9030008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.103+0000 7f9035ffb700 1 -- 172.21.15.3:0/4141026948 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f903c04fbd0 con 0x7f90400cfd80 2023-03-03T06:26:30.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.282+0000 7f904639a700 1 -- 172.21.15.3:0/4141026948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9040073e90 con 0x7f90400cfd80 2023-03-03T06:26:30.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.282+0000 7f9035ffb700 1 -- 172.21.15.3:0/4141026948 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f903c083020 con 0x7f90400cfd80 2023-03-03T06:26:30.287 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-03-03T06:26:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.286+0000 7f904639a700 1 -- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9024064ee0 msgr2=0x7f9024067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.286+0000 7f904639a700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9024064ee0 0x7f9024067390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9030000c00 tx=0x7f9030008030 comp rx=0 tx=0).stop 2023-03-03T06:26:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.286+0000 7f904639a700 1 -- 172.21.15.3:0/4141026948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90400cfd80 msgr2=0x7f90400b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.286+0000 7f904639a700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90400cfd80 0x7f90400b65e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f903c00c5b0 tx=0x7f903c00c8c0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.286+0000 7f904639a700 1 -- 172.21.15.3:0/4141026948 shutdown_connections 2023-03-03T06:26:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.286+0000 7f904639a700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9024064ee0 0x7f9024067390 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.286+0000 7f904639a700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f904015d6e0 0x7f90400c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.286+0000 7f904639a700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9040156ba0 0x7f90400b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.286+0000 7f904639a700 1 --2- 172.21.15.3:0/4141026948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90400cfd80 0x7f90400b65e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.286+0000 7f904639a700 1 -- 172.21.15.3:0/4141026948 >> 172.21.15.3:0/4141026948 conn(0x7f904006e5c0 msgr2=0x7f90400d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:30.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.286+0000 7f904639a700 1 -- 172.21.15.3:0/4141026948 shutdown_connections 2023-03-03T06:26:30.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.286+0000 7f904639a700 1 -- 172.21.15.3:0/4141026948 wait complete. 2023-03-03T06:26:30.326 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-03T06:26:30.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.530+0000 7f421d412700 1 Processor -- start 2023-03-03T06:26:30.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.530+0000 7f421d412700 1 -- start start 2023-03-03T06:26:30.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.534+0000 7f421d412700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4218156ba0 0x7f4218071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.534+0000 7f421d412700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42180d0b30 0x7f42180d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.534+0000 7f421d412700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42180cf3b0 con 0x7f421815d6e0 2023-03-03T06:26:30.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.534+0000 7f421d412700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42180d14f0 con 0x7f4218156ba0 2023-03-03T06:26:30.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.534+0000 7f421d412700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4218156f70 con 0x7f42180d0b30 2023-03-03T06:26:30.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.534+0000 7f4217fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4218156ba0 0x7f4218071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.534+0000 7f4217fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4218156ba0 0x7f4218071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41878/0 (socket says 172.21.15.3:41878) 2023-03-03T06:26:30.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.534+0000 7f4217fff700 1 -- 172.21.15.3:0/2937472186 learned_addr learned my addr 172.21.15.3:0/2937472186 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:30.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f421cc11700 1 --2- 172.21.15.3:0/2937472186 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42180d0b30 0x7f42180d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f4217fff700 1 -- 172.21.15.3:0/2937472186 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42180d0b30 msgr2=0x7f42180d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:30.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f4217fff700 1 --2- 172.21.15.3:0/2937472186 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42180d0b30 0x7f42180d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f4217fff700 1 -- 172.21.15.3:0/2937472186 >> v1:172.21.15.32:6789/0 conn(0x7f421815d6e0 legacy=0x7f42180cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f4217fff700 1 -- 172.21.15.3:0/2937472186 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42180cd320 con 0x7f4218156ba0 2023-03-03T06:26:30.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f4217fff700 1 --2- 172.21.15.3:0/2937472186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4218156ba0 0x7f4218071280 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4208009e50 tx=0x7f420800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101247dc000eabbb server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:30.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f42177fe700 1 -- 172.21.15.3:0/2937472186 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420800e070 con 0x7f4218156ba0 2023-03-03T06:26:30.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f42177fe700 1 -- 172.21.15.3:0/2937472186 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420800b9c0 con 0x7f4218156ba0 2023-03-03T06:26:30.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f42177fe700 1 -- 172.21.15.3:0/2937472186 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420800bd70 con 0x7f4218156ba0 2023-03-03T06:26:30.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f421d412700 1 -- 172.21.15.3:0/2937472186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4218156ba0 msgr2=0x7f4218071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f421d412700 1 --2- 172.21.15.3:0/2937472186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4218156ba0 0x7f4218071280 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4208009e50 tx=0x7f420800b330 comp rx=0 tx=0).stop 2023-03-03T06:26:30.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f421d412700 1 -- 172.21.15.3:0/2937472186 shutdown_connections 2023-03-03T06:26:30.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f421d412700 1 --2- 172.21.15.3:0/2937472186 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42180d0b30 0x7f42180d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f421d412700 1 --2- 172.21.15.3:0/2937472186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4218156ba0 0x7f4218071280 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.538+0000 7f421d412700 1 -- 172.21.15.3:0/2937472186 >> 172.21.15.3:0/2937472186 conn(0x7f421806e5c0 msgr2=0x7f421806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:30.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.542+0000 7f421d412700 1 -- 172.21.15.3:0/2937472186 shutdown_connections 2023-03-03T06:26:30.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.542+0000 7f421d412700 1 -- 172.21.15.3:0/2937472186 wait complete. 2023-03-03T06:26:30.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f421d412700 1 Processor -- start 2023-03-03T06:26:30.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f421d412700 1 -- start start 2023-03-03T06:26:30.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f421d412700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42180d0b30 0x7f42180c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f421d412700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4218156ba0 0x7f4218157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f421d412700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f421815d6e0 0x7f4218157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f421d412700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42180b7c40 con 0x7f4218156ba0 2023-03-03T06:26:30.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f421d412700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42180b7030 con 0x7f421815d6e0 2023-03-03T06:26:30.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f421d412700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42180b7310 con 0x7f42180d0b30 2023-03-03T06:26:30.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f420f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4218156ba0 0x7f4218157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f420f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4218156ba0 0x7f4218157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41884/0 (socket says 172.21.15.3:41884) 2023-03-03T06:26:30.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f420f7fe700 1 -- 172.21.15.3:0/2592869055 learned_addr learned my addr 172.21.15.3:0/2592869055 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:30.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f421cc11700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f421815d6e0 0x7f4218157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f4217fff700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42180d0b30 0x7f42180c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f420f7fe700 1 -- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42180d0b30 msgr2=0x7f42180c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f420f7fe700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42180d0b30 0x7f42180c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f420f7fe700 1 -- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f421815d6e0 msgr2=0x7f4218157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f420f7fe700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f421815d6e0 0x7f4218157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.546+0000 7f420f7fe700 1 -- 172.21.15.3:0/2592869055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4208009b30 con 0x7f4218156ba0 2023-03-03T06:26:30.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.550+0000 7f420f7fe700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4218156ba0 0x7f4218157890 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f420000c500 tx=0x7f420000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:30.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.550+0000 7f42157fa700 1 -- 172.21.15.3:0/2592869055 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000052d0 con 0x7f4218156ba0 2023-03-03T06:26:30.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.550+0000 7f42157fa700 1 -- 172.21.15.3:0/2592869055 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4200005430 con 0x7f4218156ba0 2023-03-03T06:26:30.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.550+0000 7f42157fa700 1 -- 172.21.15.3:0/2592869055 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000055a0 con 0x7f4218156ba0 2023-03-03T06:26:30.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.550+0000 7f421d412700 1 -- 172.21.15.3:0/2592869055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42180cd8e0 con 0x7f4218156ba0 2023-03-03T06:26:30.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.550+0000 7f421d412700 1 -- 172.21.15.3:0/2592869055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42180cd600 con 0x7f4218156ba0 2023-03-03T06:26:30.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.550+0000 7f421d412700 1 -- 172.21.15.3:0/2592869055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42180717c0 con 0x7f4218156ba0 2023-03-03T06:26:30.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.554+0000 7f42157fa700 1 -- 172.21.15.3:0/2592869055 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4200019030 con 0x7f4218156ba0 2023-03-03T06:26:30.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.558+0000 7f42157fa700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f41f8064f10 0x7f41f80673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.558+0000 7f42157fa700 1 -- 172.21.15.3:0/2592869055 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4200081520 con 0x7f4218156ba0 2023-03-03T06:26:30.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.558+0000 7f4217fff700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f41f8064f10 0x7f41f80673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.558+0000 7f4217fff700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f41f8064f10 0x7f41f80673c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f420800b370 tx=0x7f4208009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:30.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.562+0000 7f42157fa700 1 -- 172.21.15.3:0/2592869055 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f420004f2a0 con 0x7f4218156ba0 2023-03-03T06:26:30.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.738+0000 7f421d412700 1 -- 172.21.15.3:0/2592869055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4218073e90 con 0x7f4218156ba0 2023-03-03T06:26:30.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.742+0000 7f42157fa700 1 -- 172.21.15.3:0/2592869055 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f42000528c0 con 0x7f4218156ba0 2023-03-03T06:26:30.746 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:30.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.746+0000 7f421d412700 1 -- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f41f8064f10 msgr2=0x7f41f80673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.746+0000 7f421d412700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f41f8064f10 0x7f41f80673c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f420800b370 tx=0x7f4208009000 comp rx=0 tx=0).stop 2023-03-03T06:26:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.746+0000 7f421d412700 1 -- 172.21.15.3:0/2592869055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4218156ba0 msgr2=0x7f4218157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.746+0000 7f421d412700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4218156ba0 0x7f4218157890 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f420000c500 tx=0x7f420000c810 comp rx=0 tx=0).stop 2023-03-03T06:26:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.746+0000 7f421d412700 1 -- 172.21.15.3:0/2592869055 shutdown_connections 2023-03-03T06:26:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.746+0000 7f421d412700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f41f8064f10 0x7f41f80673c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.746+0000 7f421d412700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f421815d6e0 0x7f4218157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.746+0000 7f421d412700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4218156ba0 0x7f4218157890 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.746+0000 7f421d412700 1 --2- 172.21.15.3:0/2592869055 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f42180d0b30 0x7f42180c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.746+0000 7f421d412700 1 -- 172.21.15.3:0/2592869055 >> 172.21.15.3:0/2592869055 conn(0x7f421806e5c0 msgr2=0x7f421815f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:30.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.746+0000 7f421d412700 1 -- 172.21.15.3:0/2592869055 shutdown_connections 2023-03-03T06:26:30.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.746+0000 7f421d412700 1 -- 172.21.15.3:0/2592869055 wait complete. 2023-03-03T06:26:30.785 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-03T06:26:30.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.978+0000 7fdbdb59e700 1 Processor -- start 2023-03-03T06:26:30.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.978+0000 7fdbdb59e700 1 -- start start 2023-03-03T06:26:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.978+0000 7fdbdb59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdbdc15dec0 0x7fdbdc15e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbdb59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdbdc0cb180 0x7fdbdc0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbdb59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbdc0c9a20 con 0x7fdbdc0adf20 2023-03-03T06:26:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbdb59e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbdc0cbb60 con 0x7fdbdc15dec0 2023-03-03T06:26:30.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbdb59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbdc0ae2f0 con 0x7fdbdc0cb180 2023-03-03T06:26:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbd9d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdbdc15dec0 0x7fdbdc15e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbda59c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdbdc0adf20 0x7fdbdc0b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43072/0 (socket says 172.21.15.3:43072) 2023-03-03T06:26:30.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbd9d9b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdbdc15dec0 0x7fdbdc15e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41890/0 (socket says 172.21.15.3:41890) 2023-03-03T06:26:30.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbda59c700 1 -- 172.21.15.3:0/1743203935 learned_addr learned my addr 172.21.15.3:0/1743203935 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:30.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbdad9d700 1 --2- 172.21.15.3:0/1743203935 >> v2:172.21.15.32:3300/0 conn(0x7fdbdc0cb180 0x7fdbdc0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbd9d9b700 1 -- 172.21.15.3:0/1743203935 >> v2:172.21.15.32:3300/0 conn(0x7fdbdc0cb180 msgr2=0x7fdbdc0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:30.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbd9d9b700 1 --2- 172.21.15.3:0/1743203935 >> v2:172.21.15.32:3300/0 conn(0x7fdbdc0cb180 0x7fdbdc0cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbd9d9b700 1 -- 172.21.15.3:0/1743203935 >> v1:172.21.15.3:6789/0 conn(0x7fdbdc0adf20 legacy=0x7fdbdc0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbd9d9b700 1 -- 172.21.15.3:0/1743203935 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc0c9fe0 con 0x7fdbdc15dec0 2023-03-03T06:26:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbd9d9b700 1 --2- 172.21.15.3:0/1743203935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbdc15dec0 0x7fdbdc15e310 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0cbf90 tx=0x7fdbd0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85866a9c61a198c server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbd959a700 1 -- 172.21.15.3:0/1743203935 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0011830 con 0x7fdbdc15dec0 2023-03-03T06:26:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbd959a700 1 -- 172.21.15.3:0/1743203935 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd0011990 con 0x7fdbdc15dec0 2023-03-03T06:26:30.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.982+0000 7fdbd959a700 1 -- 172.21.15.3:0/1743203935 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0011d40 con 0x7fdbdc15dec0 2023-03-03T06:26:30.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.986+0000 7fdbdb59e700 1 -- 172.21.15.3:0/1743203935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbdc15dec0 msgr2=0x7fdbdc15e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.986+0000 7fdbdb59e700 1 --2- 172.21.15.3:0/1743203935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbdc15dec0 0x7fdbdc15e310 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0cbf90 tx=0x7fdbd0009b30 comp rx=0 tx=0).stop 2023-03-03T06:26:30.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.986+0000 7fdbdb59e700 1 -- 172.21.15.3:0/1743203935 shutdown_connections 2023-03-03T06:26:30.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.986+0000 7fdbdb59e700 1 --2- 172.21.15.3:0/1743203935 >> v2:172.21.15.32:3300/0 conn(0x7fdbdc0cb180 0x7fdbdc0cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.986+0000 7fdbdb59e700 1 --2- 172.21.15.3:0/1743203935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbdc15dec0 0x7fdbdc15e310 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.986+0000 7fdbdb59e700 1 -- 172.21.15.3:0/1743203935 >> 172.21.15.3:0/1743203935 conn(0x7fdbdc070a30 msgr2=0x7fdbdc070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:30.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.986+0000 7fdbdb59e700 1 -- 172.21.15.3:0/1743203935 shutdown_connections 2023-03-03T06:26:30.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.986+0000 7fdbdb59e700 1 -- 172.21.15.3:0/1743203935 wait complete. 2023-03-03T06:26:30.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbdb59e700 1 Processor -- start 2023-03-03T06:26:30.994 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbdb59e700 1 -- start start 2023-03-03T06:26:30.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbdb59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbdc0adf20 0x7fdbdc15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbdb59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbdc0cb180 0x7fdbdc0acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbdb59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbdc15dec0 0x7fdbdc0c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:30.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbdb59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc156460 con 0x7fdbdc0cb180 2023-03-03T06:26:30.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbdb59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc0b1b30 con 0x7fdbdc0adf20 2023-03-03T06:26:30.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbdb59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc0b1e10 con 0x7fdbdc15dec0 2023-03-03T06:26:30.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbd9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbdc0cb180 0x7fdbdc0acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbd9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbdc0cb180 0x7fdbdc0acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41900/0 (socket says 172.21.15.3:41900) 2023-03-03T06:26:30.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbd9d9b700 1 -- 172.21.15.3:0/3534041292 learned_addr learned my addr 172.21.15.3:0/3534041292 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:30.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbda59c700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbdc0adf20 0x7fdbdc15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbdad9d700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbdc15dec0 0x7fdbdc0c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbd9d9b700 1 -- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbdc15dec0 msgr2=0x7fdbdc0c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbd9d9b700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbdc15dec0 0x7fdbdc0c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbd9d9b700 1 -- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbdc0adf20 msgr2=0x7fdbdc15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbd9d9b700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbdc0adf20 0x7fdbdc15cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbd9d9b700 1 -- 172.21.15.3:0/3534041292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00094b0 con 0x7fdbdc0cb180 2023-03-03T06:26:30.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbdad9d700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbdc15dec0 0x7fdbdc0c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:30.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.990+0000 7fdbda59c700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbdc0adf20 0x7fdbdc15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:31.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.994+0000 7fdbd9d9b700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbdc0cb180 0x7fdbdc0acb80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0014040 tx=0x7fdbd0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:31.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.994+0000 7fdbcaffd700 1 -- 172.21.15.3:0/3534041292 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd00038c0 con 0x7fdbdc0cb180 2023-03-03T06:26:31.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.994+0000 7fdbcaffd700 1 -- 172.21.15.3:0/3534041292 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd001e070 con 0x7fdbdc0cb180 2023-03-03T06:26:31.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.994+0000 7fdbdb59e700 1 -- 172.21.15.3:0/3534041292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbdc15e910 con 0x7fdbdc0cb180 2023-03-03T06:26:31.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.994+0000 7fdbcaffd700 1 -- 172.21.15.3:0/3534041292 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0003c80 con 0x7fdbdc0cb180 2023-03-03T06:26:31.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.994+0000 7fdbdb59e700 1 -- 172.21.15.3:0/3534041292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbdc15a7d0 con 0x7fdbdc0cb180 2023-03-03T06:26:31.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.994+0000 7fdbdb59e700 1 -- 172.21.15.3:0/3534041292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbdc15ebf0 con 0x7fdbdc0cb180 2023-03-03T06:26:31.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.998+0000 7fdbcaffd700 1 -- 172.21.15.3:0/3534041292 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fdbd000f050 con 0x7fdbdc0cb180 2023-03-03T06:26:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.998+0000 7fdbcaffd700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdbbc064ee0 0x7fdbbc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.998+0000 7fdbcaffd700 1 -- 172.21.15.3:0/3534041292 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdbd0020070 con 0x7fdbdc0cb180 2023-03-03T06:26:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:30.998+0000 7fdbda59c700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdbbc064ee0 0x7fdbbc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:31.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.002+0000 7fdbda59c700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdbbc064ee0 0x7fdbbc067390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc007fd0 tx=0x7fdbcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:31.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.006+0000 7fdbcaffd700 1 -- 172.21.15.3:0/3534041292 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fdbd00515e0 con 0x7fdbdc0cb180 2023-03-03T06:26:31.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.194+0000 7fdbdb59e700 1 -- 172.21.15.3:0/3534041292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdbdc066cc0 con 0x7fdbdc0cb180 2023-03-03T06:26:31.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.194+0000 7fdbcaffd700 1 -- 172.21.15.3:0/3534041292 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdbd0015080 con 0x7fdbdc0cb180 2023-03-03T06:26:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.202+0000 7fdbdb59e700 1 -- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdbbc064ee0 msgr2=0x7fdbbc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.202+0000 7fdbdb59e700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdbbc064ee0 0x7fdbbc067390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc007fd0 tx=0x7fdbcc009040 comp rx=0 tx=0).stop 2023-03-03T06:26:31.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.202+0000 7fdbdb59e700 1 -- 172.21.15.3:0/3534041292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbdc0cb180 msgr2=0x7fdbdc0acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:31.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.202+0000 7fdbdb59e700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbdc0cb180 0x7fdbdc0acb80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0014040 tx=0x7fdbd0004770 comp rx=0 tx=0).stop 2023-03-03T06:26:31.208 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.202+0000 7fdbdb59e700 1 -- 172.21.15.3:0/3534041292 shutdown_connections 2023-03-03T06:26:31.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.202+0000 7fdbdb59e700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdbbc064ee0 0x7fdbbc067390 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:31.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.202+0000 7fdbdb59e700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdbdc15dec0 0x7fdbdc0c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:31.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.202+0000 7fdbdb59e700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbdc0cb180 0x7fdbdc0acb80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:31.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.202+0000 7fdbdb59e700 1 --2- 172.21.15.3:0/3534041292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbdc0adf20 0x7fdbdc15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:31.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.202+0000 7fdbdb59e700 1 -- 172.21.15.3:0/3534041292 >> 172.21.15.3:0/3534041292 conn(0x7fdbdc070a30 msgr2=0x7fdbdc0ca470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:31.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.202+0000 7fdbdb59e700 1 -- 172.21.15.3:0/3534041292 shutdown_connections 2023-03-03T06:26:31.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.202+0000 7fdbdb59e700 1 -- 172.21.15.3:0/3534041292 wait complete. 2023-03-03T06:26:31.212 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2023-03-03T06:26:31.242 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-03-03T06:26:31.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:31.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-03-03T06:26:31.329 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-03-03T06:26:31.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:26:31.330 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-03-03T06:26:31.355 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-03-03T06:26:31.355 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-03T06:26:31.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.562+0000 7f709f4c4700 1 Processor -- start 2023-03-03T06:26:31.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.562+0000 7f709f4c4700 1 -- start start 2023-03-03T06:26:31.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.562+0000 7f709f4c4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7098160880 0x7f70980c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:31.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.562+0000 7f709f4c4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70980aea70 0x7f70980ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:31.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.562+0000 7f709f4c4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70980ce190 con 0x7f70980adcf0 2023-03-03T06:26:31.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.562+0000 7f709f4c4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70980ae190 con 0x7f70980aea70 2023-03-03T06:26:31.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.562+0000 7f709f4c4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c99d0 con 0x7f7098160880 2023-03-03T06:26:31.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709dcc1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70980aea70 0x7f70980ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:31.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709dcc1700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f70980aea70 0x7f70980ad7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41902/0 (socket says 172.21.15.3:41902) 2023-03-03T06:26:31.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709dcc1700 1 -- 172.21.15.3:0/1342572350 learned_addr learned my addr 172.21.15.3:0/1342572350 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:31.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709e4c2700 1 --2- 172.21.15.3:0/1342572350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7098160880 0x7f70980c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:31.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709dcc1700 1 -- 172.21.15.3:0/1342572350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7098160880 msgr2=0x7f70980c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:31.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709dcc1700 1 --2- 172.21.15.3:0/1342572350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7098160880 0x7f70980c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:31.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709dcc1700 1 -- 172.21.15.3:0/1342572350 >> v1:172.21.15.32:6789/0 conn(0x7f70980adcf0 legacy=0x7f70981644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:31.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709dcc1700 1 -- 172.21.15.3:0/1342572350 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70980cd580 con 0x7f70980aea70 2023-03-03T06:26:31.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709dcc1700 1 --2- 172.21.15.3:0/1342572350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70980aea70 0x7f70980ad7b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7088009910 tx=0x7f7088009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9141617473334f server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:31.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709d4c0700 1 -- 172.21.15.3:0/1342572350 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088011830 con 0x7f70980aea70 2023-03-03T06:26:31.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709d4c0700 1 -- 172.21.15.3:0/1342572350 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7088011990 con 0x7f70980aea70 2023-03-03T06:26:31.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709d4c0700 1 -- 172.21.15.3:0/1342572350 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088011d40 con 0x7f70980aea70 2023-03-03T06:26:31.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709f4c4700 1 -- 172.21.15.3:0/1342572350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70980aea70 msgr2=0x7f70980ad7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:31.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.566+0000 7f709f4c4700 1 --2- 172.21.15.3:0/1342572350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70980aea70 0x7f70980ad7b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7088009910 tx=0x7f7088009c20 comp rx=0 tx=0).stop 2023-03-03T06:26:31.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.570+0000 7f709f4c4700 1 -- 172.21.15.3:0/1342572350 shutdown_connections 2023-03-03T06:26:31.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.570+0000 7f709f4c4700 1 --2- 172.21.15.3:0/1342572350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70980aea70 0x7f70980ad7b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:31.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.570+0000 7f709f4c4700 1 --2- 172.21.15.3:0/1342572350 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7098160880 0x7f70980c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:31.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.570+0000 7f709f4c4700 1 -- 172.21.15.3:0/1342572350 >> 172.21.15.3:0/1342572350 conn(0x7f7098070a30 msgr2=0x7f7098070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:31.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.570+0000 7f709f4c4700 1 -- 172.21.15.3:0/1342572350 shutdown_connections 2023-03-03T06:26:31.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.570+0000 7f709f4c4700 1 -- 172.21.15.3:0/1342572350 wait complete. 2023-03-03T06:26:31.576 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.570+0000 7f709f4c4700 1 Processor -- start 2023-03-03T06:26:31.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.570+0000 7f709f4c4700 1 -- start start 2023-03-03T06:26:31.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709f4c4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70980adcf0 0x7f70981f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:31.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709f4c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70980aea70 0x7f709806d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:31.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709f4c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7098160880 0x7f709806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:31.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709f4c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980b3fd0 con 0x7f70980aea70 2023-03-03T06:26:31.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709f4c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980b1760 con 0x7f7098160880 2023-03-03T06:26:31.579 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709f4c4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70980b1a40 con 0x7f70980adcf0 2023-03-03T06:26:31.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709dcc1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70980aea70 0x7f709806d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:31.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709dcc1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70980aea70 0x7f709806d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41906/0 (socket says 172.21.15.3:41906) 2023-03-03T06:26:31.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709dcc1700 1 -- 172.21.15.3:0/1675540573 learned_addr learned my addr 172.21.15.3:0/1675540573 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:31.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709e4c2700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70980adcf0 0x7f70981f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:31.580 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709ecc3700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7098160880 0x7f709806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:31.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709dcc1700 1 -- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70980adcf0 msgr2=0x7f70981f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:31.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709dcc1700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70980adcf0 0x7f70981f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:31.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709dcc1700 1 -- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7098160880 msgr2=0x7f709806d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:31.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709dcc1700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7098160880 0x7f709806d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:31.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709dcc1700 1 -- 172.21.15.3:0/1675540573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7088009570 con 0x7f70980aea70 2023-03-03T06:26:31.581 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709ecc3700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7098160880 0x7f709806d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:31.582 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709dcc1700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70980aea70 0x7f709806d080 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7088013000 tx=0x7f7088004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:31.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f708effd700 1 -- 172.21.15.3:0/1675540573 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088004590 con 0x7f70980aea70 2023-03-03T06:26:31.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f708effd700 1 -- 172.21.15.3:0/1675540573 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708801e070 con 0x7f70980aea70 2023-03-03T06:26:31.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f708effd700 1 -- 172.21.15.3:0/1675540573 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088003900 con 0x7f70980aea70 2023-03-03T06:26:31.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709f4c4700 1 -- 172.21.15.3:0/1675540573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70980cdb40 con 0x7f70980aea70 2023-03-03T06:26:31.583 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.574+0000 7f709f4c4700 1 -- 172.21.15.3:0/1675540573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70980cd860 con 0x7f70980aea70 2023-03-03T06:26:31.584 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.578+0000 7f709f4c4700 1 -- 172.21.15.3:0/1675540573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70980cde20 con 0x7f70980aea70 2023-03-03T06:26:31.586 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.582+0000 7f708effd700 1 -- 172.21.15.3:0/1675540573 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f708800f050 con 0x7f70980aea70 2023-03-03T06:26:31.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.582+0000 7f708effd700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f707c064ee0 0x7f707c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:31.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.582+0000 7f708effd700 1 -- 172.21.15.3:0/1675540573 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7088082fd0 con 0x7f70980aea70 2023-03-03T06:26:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.582+0000 7f709e4c2700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f707c064ee0 0x7f707c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.586+0000 7f709e4c2700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f707c064ee0 0x7f707c067390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7094009ae0 tx=0x7f7094008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:31.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.590+0000 7f708effd700 1 -- 172.21.15.3:0/1675540573 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7088050d50 con 0x7f70980aea70 2023-03-03T06:26:31.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.806+0000 7f709f4c4700 1 -- 172.21.15.3:0/1675540573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7098073e90 con 0x7f70980aea70 2023-03-03T06:26:31.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.806+0000 7f708effd700 1 -- 172.21.15.3:0/1675540573 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4635 (secure 0 0 0) 0x7f7088020090 con 0x7f70980aea70 2023-03-03T06:26:31.813 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:31.813 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2744699972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2744699972},{"type":"v1","addr":"172.21.15.3:6834","nonce":2744699972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/708054272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":708054272},{"type":"v1","addr":"172.21.15.3:6835","nonce":708054272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2935923413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2935923413},{"type":"v1","addr":"172.21.15.32:6837","nonce":2935923413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-03T06:25:13.798672+0000","modified":"2023-03-03T06:25:41.755082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/691220172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":691220172},{"type":"v1","addr":"172.21.15.32:6836","nonce":691220172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-03-03T06:26:31.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.810+0000 7f709f4c4700 1 -- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f707c064ee0 msgr2=0x7f707c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:31.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.810+0000 7f709f4c4700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f707c064ee0 0x7f707c067390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7094009ae0 tx=0x7f7094008040 comp rx=0 tx=0).stop 2023-03-03T06:26:31.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.810+0000 7f709f4c4700 1 -- 172.21.15.3:0/1675540573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70980aea70 msgr2=0x7f709806d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:31.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.810+0000 7f709f4c4700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70980aea70 0x7f709806d080 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7088013000 tx=0x7f7088004770 comp rx=0 tx=0).stop 2023-03-03T06:26:31.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.810+0000 7f709f4c4700 1 -- 172.21.15.3:0/1675540573 shutdown_connections 2023-03-03T06:26:31.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.810+0000 7f709f4c4700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f707c064ee0 0x7f707c067390 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:31.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.810+0000 7f709f4c4700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7098160880 0x7f709806d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:31.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.810+0000 7f709f4c4700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70980aea70 0x7f709806d080 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:31.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.810+0000 7f709f4c4700 1 --2- 172.21.15.3:0/1675540573 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f70980adcf0 0x7f70981f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:31.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.810+0000 7f709f4c4700 1 -- 172.21.15.3:0/1675540573 >> 172.21.15.3:0/1675540573 conn(0x7f7098070a30 msgr2=0x7f709806c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:31.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.810+0000 7f709f4c4700 1 -- 172.21.15.3:0/1675540573 shutdown_connections 2023-03-03T06:26:31.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:31.810+0000 7f709f4c4700 1 -- 172.21.15.3:0/1675540573 wait complete. 2023-03-03T06:26:31.821 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2023-03-03T06:26:31.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:32.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.058+0000 7f6c71451700 1 Processor -- start 2023-03-03T06:26:32.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.058+0000 7f6c71451700 1 -- start start 2023-03-03T06:26:32.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c71451700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c6c0b2370 0x7f6c6c160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c71451700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6c6c0caf40 0x7f6c6c0ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c71451700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c0ce1c0 con 0x7f6c6c145e20 2023-03-03T06:26:32.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c71451700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6c160f30 con 0x7f6c6c0caf40 2023-03-03T06:26:32.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c71451700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6c160970 con 0x7f6c6c0b2370 2023-03-03T06:26:32.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c6bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6c6c0caf40 0x7f6c6c0ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c6bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6c6c0caf40 0x7f6c6c0ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41910/0 (socket says 172.21.15.3:41910) 2023-03-03T06:26:32.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c6bfff700 1 -- 172.21.15.3:0/1720473603 learned_addr learned my addr 172.21.15.3:0/1720473603 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:32.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c6a7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6c6c145e20 0x7f6c6c0ca830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43084/0 (socket says 172.21.15.3:43084) 2023-03-03T06:26:32.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c6affd700 1 --2- 172.21.15.3:0/1720473603 >> v2:172.21.15.32:3300/0 conn(0x7f6c6c0b2370 0x7f6c6c160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c6bfff700 1 -- 172.21.15.3:0/1720473603 >> v2:172.21.15.32:3300/0 conn(0x7f6c6c0b2370 msgr2=0x7f6c6c160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c6bfff700 1 --2- 172.21.15.3:0/1720473603 >> v2:172.21.15.32:3300/0 conn(0x7f6c6c0b2370 0x7f6c6c160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c6bfff700 1 -- 172.21.15.3:0/1720473603 >> v1:172.21.15.3:6789/0 conn(0x7f6c6c145e20 legacy=0x7f6c6c0ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c6bfff700 1 -- 172.21.15.3:0/1720473603 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c160c50 con 0x7f6c6c0caf40 2023-03-03T06:26:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.062+0000 7f6c6bfff700 1 --2- 172.21.15.3:0/1720473603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c6c0caf40 0x7f6c6c0ce650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0ccbc0 tx=0x7f6c6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e6502e579b5529 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.066+0000 7f6c69ffb700 1 -- 172.21.15.3:0/1720473603 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400e070 con 0x7f6c6c0caf40 2023-03-03T06:26:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.066+0000 7f6c69ffb700 1 -- 172.21.15.3:0/1720473603 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6400b9c0 con 0x7f6c6c0caf40 2023-03-03T06:26:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.066+0000 7f6c69ffb700 1 -- 172.21.15.3:0/1720473603 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400bd70 con 0x7f6c6c0caf40 2023-03-03T06:26:32.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.066+0000 7f6c71451700 1 -- 172.21.15.3:0/1720473603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c6c0caf40 msgr2=0x7f6c6c0ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.066+0000 7f6c71451700 1 --2- 172.21.15.3:0/1720473603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c6c0caf40 0x7f6c6c0ce650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0ccbc0 tx=0x7f6c6400b330 comp rx=0 tx=0).stop 2023-03-03T06:26:32.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.066+0000 7f6c71451700 1 -- 172.21.15.3:0/1720473603 shutdown_connections 2023-03-03T06:26:32.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.066+0000 7f6c71451700 1 --2- 172.21.15.3:0/1720473603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c6c0caf40 0x7f6c6c0ce650 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.066+0000 7f6c71451700 1 --2- 172.21.15.3:0/1720473603 >> v2:172.21.15.32:3300/0 conn(0x7f6c6c0b2370 0x7f6c6c160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.066+0000 7f6c71451700 1 -- 172.21.15.3:0/1720473603 >> 172.21.15.3:0/1720473603 conn(0x7f6c6c06b580 msgr2=0x7f6c6c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:32.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.066+0000 7f6c71451700 1 -- 172.21.15.3:0/1720473603 shutdown_connections 2023-03-03T06:26:32.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.066+0000 7f6c71451700 1 -- 172.21.15.3:0/1720473603 wait complete. 2023-03-03T06:26:32.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.070+0000 7f6c71451700 1 Processor -- start 2023-03-03T06:26:32.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.070+0000 7f6c71451700 1 -- start start 2023-03-03T06:26:32.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.070+0000 7f6c71451700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c6c0b2370 0x7f6c6c1f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.070+0000 7f6c71451700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c6c0caf40 0x7f6c6c158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.070+0000 7f6c71451700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c6c145e20 0x7f6c6c0c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.070+0000 7f6c71451700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c1f2bf0 con 0x7f6c6c0caf40 2023-03-03T06:26:32.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.070+0000 7f6c71451700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c1f61f0 con 0x7f6c6c145e20 2023-03-03T06:26:32.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.070+0000 7f6c71451700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c1f1f70 con 0x7f6c6c0b2370 2023-03-03T06:26:32.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c6a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c6c0caf40 0x7f6c6c158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c6a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c6c0caf40 0x7f6c6c158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41912/0 (socket says 172.21.15.3:41912) 2023-03-03T06:26:32.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c6a7fc700 1 -- 172.21.15.3:0/111948012 learned_addr learned my addr 172.21.15.3:0/111948012 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:32.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c6affd700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c6c0b2370 0x7f6c6c1f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c6bfff700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c6c145e20 0x7f6c6c0c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c6a7fc700 1 -- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c6c0b2370 msgr2=0x7f6c6c1f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c6a7fc700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c6c0b2370 0x7f6c6c1f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c6a7fc700 1 -- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c6c145e20 msgr2=0x7f6c6c0c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c6a7fc700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c6c145e20 0x7f6c6c0c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c6a7fc700 1 -- 172.21.15.3:0/111948012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c64009b30 con 0x7f6c6c0caf40 2023-03-03T06:26:32.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c6bfff700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c6c145e20 0x7f6c6c0c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:32.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c6a7fc700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c6c0caf40 0x7f6c6c158760 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000c460 tx=0x7f6c6000c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:32.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c537fe700 1 -- 172.21.15.3:0/111948012 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60005cd0 con 0x7f6c6c0caf40 2023-03-03T06:26:32.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c537fe700 1 -- 172.21.15.3:0/111948012 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c60005e30 con 0x7f6c6c0caf40 2023-03-03T06:26:32.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c537fe700 1 -- 172.21.15.3:0/111948012 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c600072d0 con 0x7f6c6c0caf40 2023-03-03T06:26:32.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c71451700 1 -- 172.21.15.3:0/111948012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6c0b2740 con 0x7f6c6c0caf40 2023-03-03T06:26:32.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.074+0000 7f6c71451700 1 -- 172.21.15.3:0/111948012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6c0cde50 con 0x7f6c6c0caf40 2023-03-03T06:26:32.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.078+0000 7f6c71451700 1 -- 172.21.15.3:0/111948012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6c160540 con 0x7f6c6c0caf40 2023-03-03T06:26:32.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.078+0000 7f6c537fe700 1 -- 172.21.15.3:0/111948012 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6c60019030 con 0x7f6c6c0caf40 2023-03-03T06:26:32.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.082+0000 7f6c537fe700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6c58064ee0 0x7f6c58067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.086 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.082+0000 7f6c537fe700 1 -- 172.21.15.3:0/111948012 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6c600813d0 con 0x7f6c6c0caf40 2023-03-03T06:26:32.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.082+0000 7f6c6affd700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6c58064ee0 0x7f6c58067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.082+0000 7f6c6affd700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6c58064ee0 0x7f6c58067390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c1584a0 tx=0x7f6c5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:32.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.086+0000 7f6c537fe700 1 -- 172.21.15.3:0/111948012 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6c6004a400 con 0x7f6c6c0caf40 2023-03-03T06:26:32.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.262+0000 7f6c71451700 1 -- 172.21.15.3:0/111948012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c6c04e180 con 0x7f6c6c0caf40 2023-03-03T06:26:32.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.266+0000 7f6c537fe700 1 -- 172.21.15.3:0/111948012 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f6c6004f200 con 0x7f6c6c0caf40 2023-03-03T06:26:32.270 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:32.271 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:53.877945+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:32.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.266+0000 7f6c71451700 1 -- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6c58064ee0 msgr2=0x7f6c58067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.266+0000 7f6c71451700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6c58064ee0 0x7f6c58067390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c1584a0 tx=0x7f6c5c008040 comp rx=0 tx=0).stop 2023-03-03T06:26:32.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.266+0000 7f6c71451700 1 -- 172.21.15.3:0/111948012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c6c0caf40 msgr2=0x7f6c6c158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.266+0000 7f6c71451700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c6c0caf40 0x7f6c6c158760 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000c460 tx=0x7f6c6000c770 comp rx=0 tx=0).stop 2023-03-03T06:26:32.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.266+0000 7f6c71451700 1 -- 172.21.15.3:0/111948012 shutdown_connections 2023-03-03T06:26:32.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.266+0000 7f6c71451700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6c58064ee0 0x7f6c58067390 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.266+0000 7f6c71451700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c6c145e20 0x7f6c6c0c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.266+0000 7f6c71451700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c6c0caf40 0x7f6c6c158760 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.266+0000 7f6c71451700 1 --2- 172.21.15.3:0/111948012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6c6c0b2370 0x7f6c6c1f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.266+0000 7f6c71451700 1 -- 172.21.15.3:0/111948012 >> 172.21.15.3:0/111948012 conn(0x7f6c6c06b580 msgr2=0x7f6c6c070200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:32.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.266+0000 7f6c71451700 1 -- 172.21.15.3:0/111948012 shutdown_connections 2023-03-03T06:26:32.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.266+0000 7f6c71451700 1 -- 172.21.15.3:0/111948012 wait complete. 2023-03-03T06:26:32.308 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-03T06:25:11.468534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-03T06:25:12.813283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-03T06:25:16.382800+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-03T06:26:32.309 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:26:32.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.514+0000 7f781f59e700 1 Processor -- start 2023-03-03T06:26:32.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.514+0000 7f781f59e700 1 -- start start 2023-03-03T06:26:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781f59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7820156cc0 0x7f78200713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f78200cf610 0x7f78200cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781f59e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7820160fc0 con 0x7f782015d720 2023-03-03T06:26:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781f59e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78201600f0 con 0x7f7820156cc0 2023-03-03T06:26:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781f59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78200718e0 con 0x7f78200cf610 2023-03-03T06:26:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781dd9b700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f782015d720 0x7f78201608b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43100/0 (socket says 172.21.15.3:43100) 2023-03-03T06:26:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781dd9b700 1 -- 172.21.15.3:0/2264710962 learned_addr learned my addr 172.21.15.3:0/2264710962 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:32.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781e59c700 1 --2- 172.21.15.3:0/2264710962 >> v2:172.21.15.3:3300/0 conn(0x7f7820156cc0 0x7f78200713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781ed9d700 1 --2- 172.21.15.3:0/2264710962 >> v2:172.21.15.32:3300/0 conn(0x7f78200cf610 0x7f78200cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781d59a700 1 -- 172.21.15.3:0/2264710962 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019924429 0 0) 0x7f7820160fc0 con 0x7f782015d720 2023-03-03T06:26:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781d59a700 1 -- 172.21.15.3:0/2264710962 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78080035d0 con 0x7f782015d720 2023-03-03T06:26:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781e59c700 1 -- 172.21.15.3:0/2264710962 >> v2:172.21.15.32:3300/0 conn(0x7f78200cf610 msgr2=0x7f78200cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:32.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781e59c700 1 --2- 172.21.15.3:0/2264710962 >> v2:172.21.15.32:3300/0 conn(0x7f78200cf610 0x7f78200cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781e59c700 1 -- 172.21.15.3:0/2264710962 >> v1:172.21.15.3:6789/0 conn(0x7f782015d720 legacy=0x7f78201608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.518+0000 7f781e59c700 1 -- 172.21.15.3:0/2264710962 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78200cff50 con 0x7f7820156cc0 2023-03-03T06:26:32.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.522+0000 7f781e59c700 1 --2- 172.21.15.3:0/2264710962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7820156cc0 0x7f78200713a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f78200b2120 tx=0x7f7810009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3011887dddeaa307 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.522+0000 7f781d59a700 1 -- 172.21.15.3:0/2264710962 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810011830 con 0x7f7820156cc0 2023-03-03T06:26:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.522+0000 7f781d59a700 1 -- 172.21.15.3:0/2264710962 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7810011990 con 0x7f7820156cc0 2023-03-03T06:26:32.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.522+0000 7f781d59a700 1 -- 172.21.15.3:0/2264710962 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810011d20 con 0x7f7820156cc0 2023-03-03T06:26:32.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.522+0000 7f781f59e700 1 -- 172.21.15.3:0/2264710962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7820156cc0 msgr2=0x7f78200713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.522+0000 7f781f59e700 1 --2- 172.21.15.3:0/2264710962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7820156cc0 0x7f78200713a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f78200b2120 tx=0x7f7810009b30 comp rx=0 tx=0).stop 2023-03-03T06:26:32.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.526+0000 7f781f59e700 1 -- 172.21.15.3:0/2264710962 shutdown_connections 2023-03-03T06:26:32.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.526+0000 7f781f59e700 1 --2- 172.21.15.3:0/2264710962 >> v2:172.21.15.32:3300/0 conn(0x7f78200cf610 0x7f78200cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.526+0000 7f781f59e700 1 --2- 172.21.15.3:0/2264710962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7820156cc0 0x7f78200713a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.526+0000 7f781f59e700 1 -- 172.21.15.3:0/2264710962 >> 172.21.15.3:0/2264710962 conn(0x7f782006e5c0 msgr2=0x7f782006e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:32.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.526+0000 7f781f59e700 1 -- 172.21.15.3:0/2264710962 shutdown_connections 2023-03-03T06:26:32.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.526+0000 7f781f59e700 1 -- 172.21.15.3:0/2264710962 wait complete. 2023-03-03T06:26:32.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781f59e700 1 Processor -- start 2023-03-03T06:26:32.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781f59e700 1 -- start start 2023-03-03T06:26:32.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781f59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78200cf610 0x7f782014c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7820156cc0 0x7f78200c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f782015d720 0x7f78200cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781f59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78201f2e00 con 0x7f7820156cc0 2023-03-03T06:26:32.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781f59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782014cf30 con 0x7f78200cf610 2023-03-03T06:26:32.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781f59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78200c6f10 con 0x7f782015d720 2023-03-03T06:26:32.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7820156cc0 0x7f78200c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781dd9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7820156cc0 0x7f78200c73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41926/0 (socket says 172.21.15.3:41926) 2023-03-03T06:26:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781dd9b700 1 -- 172.21.15.3:0/1308725957 learned_addr learned my addr 172.21.15.3:0/1308725957 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781e59c700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78200cf610 0x7f782014c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781ed9d700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f782015d720 0x7f78200cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.537 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781dd9b700 1 -- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f782015d720 msgr2=0x7f78200cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781dd9b700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f782015d720 0x7f78200cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781dd9b700 1 -- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78200cf610 msgr2=0x7f782014c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781dd9b700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78200cf610 0x7f782014c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781dd9b700 1 -- 172.21.15.3:0/1308725957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78100094b0 con 0x7f7820156cc0 2023-03-03T06:26:32.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.530+0000 7f781dd9b700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7820156cc0 0x7f78200c73a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7814005930 tx=0x7f781400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:32.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.534+0000 7f780effd700 1 -- 172.21.15.3:0/1308725957 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814005d90 con 0x7f7820156cc0 2023-03-03T06:26:32.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.534+0000 7f780effd700 1 -- 172.21.15.3:0/1308725957 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78140070d0 con 0x7f7820156cc0 2023-03-03T06:26:32.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.534+0000 7f780effd700 1 -- 172.21.15.3:0/1308725957 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814007480 con 0x7f7820156cc0 2023-03-03T06:26:32.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.534+0000 7f781f59e700 1 -- 172.21.15.3:0/1308725957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78200cd700 con 0x7f7820156cc0 2023-03-03T06:26:32.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.534+0000 7f781f59e700 1 -- 172.21.15.3:0/1308725957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78200cd420 con 0x7f7820156cc0 2023-03-03T06:26:32.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.534+0000 7f781f59e700 1 -- 172.21.15.3:0/1308725957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7820160390 con 0x7f7820156cc0 2023-03-03T06:26:32.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.538+0000 7f780effd700 1 -- 172.21.15.3:0/1308725957 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7814019030 con 0x7f7820156cc0 2023-03-03T06:26:32.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.538+0000 7f780effd700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7800064ee0 0x7f7800067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.538+0000 7f780effd700 1 -- 172.21.15.3:0/1308725957 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f78140814d0 con 0x7f7820156cc0 2023-03-03T06:26:32.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.538+0000 7f781e59c700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7800064ee0 0x7f7800067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.542+0000 7f781e59c700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7800064ee0 0x7f7800067390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7810000c00 tx=0x7f7810008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:32.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.546+0000 7f780effd700 1 -- 172.21.15.3:0/1308725957 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f781404f250 con 0x7f7820156cc0 2023-03-03T06:26:32.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.722+0000 7f781f59e700 1 -- 172.21.15.3:0/1308725957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78200c7080 con 0x7f7820156cc0 2023-03-03T06:26:32.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.722+0000 7f780effd700 1 -- 172.21.15.3:0/1308725957 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7814052870 con 0x7f7820156cc0 2023-03-03T06:26:32.728 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:26:32.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.726+0000 7f781f59e700 1 -- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7800064ee0 msgr2=0x7f7800067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.726+0000 7f781f59e700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7800064ee0 0x7f7800067390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7810000c00 tx=0x7f7810008030 comp rx=0 tx=0).stop 2023-03-03T06:26:32.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.726+0000 7f781f59e700 1 -- 172.21.15.3:0/1308725957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7820156cc0 msgr2=0x7f78200c73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.726+0000 7f781f59e700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7820156cc0 0x7f78200c73a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7814005930 tx=0x7f781400c7b0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.726+0000 7f781f59e700 1 -- 172.21.15.3:0/1308725957 shutdown_connections 2023-03-03T06:26:32.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.726+0000 7f781f59e700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7800064ee0 0x7f7800067390 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.726+0000 7f781f59e700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f782015d720 0x7f78200cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.726+0000 7f781f59e700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7820156cc0 0x7f78200c73a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.726+0000 7f781f59e700 1 --2- 172.21.15.3:0/1308725957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78200cf610 0x7f782014c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.726+0000 7f781f59e700 1 -- 172.21.15.3:0/1308725957 >> 172.21.15.3:0/1308725957 conn(0x7f782006e5c0 msgr2=0x7f782015ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:32.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.726+0000 7f781f59e700 1 -- 172.21.15.3:0/1308725957 shutdown_connections 2023-03-03T06:26:32.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.726+0000 7f781f59e700 1 -- 172.21.15.3:0/1308725957 wait complete. 2023-03-03T06:26:32.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:26:32.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.978+0000 7f8fd19fb700 1 Processor -- start 2023-03-03T06:26:32.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.978+0000 7f8fd19fb700 1 -- start start 2023-03-03T06:26:32.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fd19fb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fcc06a780 0x7f8fcc0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fd19fb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8fcc0cb9a0 0x7f8fcc158400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fd19fb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc15c230 con 0x7f8fcc158940 2023-03-03T06:26:32.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fd19fb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fcc0caac0 con 0x7f8fcc0cb9a0 2023-03-03T06:26:32.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fd19fb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0cd130 con 0x7f8fcc06a780 2023-03-03T06:26:32.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fc3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8fcc0cb9a0 0x7f8fcc158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fc3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8fcc0cb9a0 0x7f8fcc158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41930/0 (socket says 172.21.15.3:41930) 2023-03-03T06:26:32.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fd11fa700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8fcc158940 0x7f8fcc0709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:58302/0 (socket says 172.21.15.3:58302) 2023-03-03T06:26:32.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fc3fff700 1 -- 172.21.15.3:0/3915444297 learned_addr learned my addr 172.21.15.3:0/3915444297 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:32.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fd09f9700 1 --2- 172.21.15.3:0/3915444297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fcc06a780 0x7f8fcc0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fc3fff700 1 -- 172.21.15.3:0/3915444297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fcc06a780 msgr2=0x7f8fcc0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:32.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fc3fff700 1 --2- 172.21.15.3:0/3915444297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fcc06a780 0x7f8fcc0cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fc3fff700 1 -- 172.21.15.3:0/3915444297 >> v1:172.21.15.32:6789/0 conn(0x7f8fcc158940 legacy=0x7f8fcc0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fc3fff700 1 -- 172.21.15.3:0/3915444297 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc0cbf60 con 0x7f8fcc0cb9a0 2023-03-03T06:26:32.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.982+0000 7f8fc37fe700 1 -- 172.21.15.3:0/3915444297 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949970686 0 0) 0x7f8fcc15c230 con 0x7f8fcc158940 2023-03-03T06:26:32.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.986+0000 7f8fc3fff700 1 --2- 172.21.15.3:0/3915444297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0cb9a0 0x7f8fcc158400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00bbb0 tx=0x7f8fbc00bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45aa4a2147299e98 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:32.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.986+0000 7f8fc37fe700 1 -- 172.21.15.3:0/3915444297 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00dd90 con 0x7f8fcc0cb9a0 2023-03-03T06:26:32.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.986+0000 7f8fc37fe700 1 -- 172.21.15.3:0/3915444297 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc010470 con 0x7f8fcc0cb9a0 2023-03-03T06:26:32.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.986+0000 7f8fc37fe700 1 -- 172.21.15.3:0/3915444297 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc010820 con 0x7f8fcc0cb9a0 2023-03-03T06:26:32.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.986+0000 7f8fd19fb700 1 -- 172.21.15.3:0/3915444297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0cb9a0 msgr2=0x7f8fcc158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:32.991 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.986+0000 7f8fd19fb700 1 --2- 172.21.15.3:0/3915444297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0cb9a0 0x7f8fcc158400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00bbb0 tx=0x7f8fbc00bec0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.986+0000 7f8fd19fb700 1 -- 172.21.15.3:0/3915444297 shutdown_connections 2023-03-03T06:26:32.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.986+0000 7f8fd19fb700 1 --2- 172.21.15.3:0/3915444297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc0cb9a0 0x7f8fcc158400 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.986+0000 7f8fd19fb700 1 --2- 172.21.15.3:0/3915444297 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fcc06a780 0x7f8fcc0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:32.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.986+0000 7f8fd19fb700 1 -- 172.21.15.3:0/3915444297 >> 172.21.15.3:0/3915444297 conn(0x7f8fcc0607c0 msgr2=0x7f8fcc060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:32.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.986+0000 7f8fd19fb700 1 -- 172.21.15.3:0/3915444297 shutdown_connections 2023-03-03T06:26:32.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.986+0000 7f8fd19fb700 1 -- 172.21.15.3:0/3915444297 wait complete. 2023-03-03T06:26:32.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.990+0000 7f8fd19fb700 1 Processor -- start 2023-03-03T06:26:32.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.990+0000 7f8fd19fb700 1 -- start start 2023-03-03T06:26:32.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.990+0000 7f8fd19fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fcc06a780 0x7f8fcc155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.990+0000 7f8fd19fb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fcc0cb9a0 0x7f8fcc1569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.990+0000 7f8fd19fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc158940 0x7f8fcc156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:32.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.990+0000 7f8fd19fb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc1558d0 con 0x7f8fcc158940 2023-03-03T06:26:32.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.990+0000 7f8fd19fb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc155d50 con 0x7f8fcc06a780 2023-03-03T06:26:32.998 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.990+0000 7f8fd19fb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0c2340 con 0x7f8fcc0cb9a0 2023-03-03T06:26:32.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fd11fa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc158940 0x7f8fcc156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:32.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fd11fa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc158940 0x7f8fcc156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41940/0 (socket says 172.21.15.3:41940) 2023-03-03T06:26:33.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fd11fa700 1 -- 172.21.15.3:0/2657104158 learned_addr learned my addr 172.21.15.3:0/2657104158 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:33.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fc3fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fcc0cb9a0 0x7f8fcc1569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fd09f9700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fcc06a780 0x7f8fcc155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fd11fa700 1 -- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fcc0cb9a0 msgr2=0x7f8fcc1569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fd11fa700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fcc0cb9a0 0x7f8fcc1569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fd11fa700 1 -- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fcc06a780 msgr2=0x7f8fcc155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fd11fa700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fcc06a780 0x7f8fcc155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fd11fa700 1 -- 172.21.15.3:0/2657104158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc00b810 con 0x7f8fcc158940 2023-03-03T06:26:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fd09f9700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fcc06a780 0x7f8fcc155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fd11fa700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc158940 0x7f8fcc156f30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4001eb0 tx=0x7f8fc400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:33.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fc17fa700 1 -- 172.21.15.3:0/2657104158 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4004ca0 con 0x7f8fcc158940 2023-03-03T06:26:33.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fc17fa700 1 -- 172.21.15.3:0/2657104158 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc4004e00 con 0x7f8fcc158940 2023-03-03T06:26:33.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fd19fb700 1 -- 172.21.15.3:0/2657104158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc15c7d0 con 0x7f8fcc158940 2023-03-03T06:26:33.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fc17fa700 1 -- 172.21.15.3:0/2657104158 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4007310 con 0x7f8fcc158940 2023-03-03T06:26:33.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.994+0000 7f8fd19fb700 1 -- 172.21.15.3:0/2657104158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc15c4f0 con 0x7f8fcc158940 2023-03-03T06:26:33.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.998+0000 7f8fd19fb700 1 -- 172.21.15.3:0/2657104158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fcc15cab0 con 0x7f8fcc158940 2023-03-03T06:26:33.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:32.998+0000 7f8fc17fa700 1 -- 172.21.15.3:0/2657104158 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8fc401a020 con 0x7f8fcc158940 2023-03-03T06:26:33.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.002+0000 7f8fc17fa700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fb8064f10 0x7f8fb80673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.002+0000 7f8fc17fa700 1 -- 172.21.15.3:0/2657104158 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8fc4081cc0 con 0x7f8fcc158940 2023-03-03T06:26:33.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.002+0000 7f8fd09f9700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fb8064f10 0x7f8fb80673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.002+0000 7f8fd09f9700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fb8064f10 0x7f8fb80673c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009be0 tx=0x7f8fc8003a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:33.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.006+0000 7f8fc17fa700 1 -- 172.21.15.3:0/2657104158 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f8fc4088080 con 0x7f8fcc158940 2023-03-03T06:26:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.186+0000 7f8fd19fb700 1 -- 172.21.15.3:0/2657104158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8fcc073e90 con 0x7f8fcc158940 2023-03-03T06:26:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.190+0000 7f8fc17fa700 1 -- 172.21.15.3:0/2657104158 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8fc404fa40 con 0x7f8fcc158940 2023-03-03T06:26:33.195 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:26:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.194+0000 7f8fd19fb700 1 -- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fb8064f10 msgr2=0x7f8fb80673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.194+0000 7f8fd19fb700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fb8064f10 0x7f8fb80673c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009be0 tx=0x7f8fc8003a80 comp rx=0 tx=0).stop 2023-03-03T06:26:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.194+0000 7f8fd19fb700 1 -- 172.21.15.3:0/2657104158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc158940 msgr2=0x7f8fcc156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.194+0000 7f8fd19fb700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc158940 0x7f8fcc156f30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4001eb0 tx=0x7f8fc400c730 comp rx=0 tx=0).stop 2023-03-03T06:26:33.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.194+0000 7f8fd19fb700 1 -- 172.21.15.3:0/2657104158 shutdown_connections 2023-03-03T06:26:33.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.194+0000 7f8fd19fb700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fb8064f10 0x7f8fb80673c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.194+0000 7f8fd19fb700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fcc158940 0x7f8fcc156f30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.194+0000 7f8fd19fb700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8fcc0cb9a0 0x7f8fcc1569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.194+0000 7f8fd19fb700 1 --2- 172.21.15.3:0/2657104158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fcc06a780 0x7f8fcc155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.194+0000 7f8fd19fb700 1 -- 172.21.15.3:0/2657104158 >> 172.21.15.3:0/2657104158 conn(0x7f8fcc0607c0 msgr2=0x7f8fcc157af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:33.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.194+0000 7f8fd19fb700 1 -- 172.21.15.3:0/2657104158 shutdown_connections 2023-03-03T06:26:33.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.194+0000 7f8fd19fb700 1 -- 172.21.15.3:0/2657104158 wait complete. 2023-03-03T06:26:33.230 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-03T06:26:33.439 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.434+0000 7fa8e9701700 1 Processor -- start 2023-03-03T06:26:33.440 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.434+0000 7fa8e9701700 1 -- start start 2023-03-03T06:26:33.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e9701700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8e4070820 0x7fa8e406b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:33.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e9701700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8e415a770 0x7fa8e415abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:33.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e9701700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e4070c70 con 0x7fa8e406b580 2023-03-03T06:26:33.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e9701700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e415ec50 con 0x7fa8e415a770 2023-03-03T06:26:33.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e9701700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e40cc990 con 0x7fa8e4070820 2023-03-03T06:26:33.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e3fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8e415a770 0x7fa8e415abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e2ffd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa8e406b580 0x7fa8e40cc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43104/0 (socket says 172.21.15.3:43104) 2023-03-03T06:26:33.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e2ffd700 1 -- 172.21.15.3:0/3169193341 learned_addr learned my addr 172.21.15.3:0/3169193341 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:33.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e27fc700 1 --2- 172.21.15.3:0/3169193341 >> v2:172.21.15.32:3300/0 conn(0x7fa8e4070820 0x7fa8e406b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e3fff700 1 -- 172.21.15.3:0/3169193341 >> v2:172.21.15.32:3300/0 conn(0x7fa8e4070820 msgr2=0x7fa8e406b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:33.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e3fff700 1 --2- 172.21.15.3:0/3169193341 >> v2:172.21.15.32:3300/0 conn(0x7fa8e4070820 0x7fa8e406b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e3fff700 1 -- 172.21.15.3:0/3169193341 >> v1:172.21.15.3:6789/0 conn(0x7fa8e406b580 legacy=0x7fa8e40cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e3fff700 1 -- 172.21.15.3:0/3169193341 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e415e7c0 con 0x7fa8e415a770 2023-03-03T06:26:33.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e1ffb700 1 -- 172.21.15.3:0/3169193341 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562741433 0 0) 0x7fa8e4070c70 con 0x7fa8e406b580 2023-03-03T06:26:33.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.438+0000 7fa8e3fff700 1 --2- 172.21.15.3:0/3169193341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e415a770 0x7fa8e415abf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4061640 tx=0x7fa8dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c544f24551a6c server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:33.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.442+0000 7fa8e1ffb700 1 -- 172.21.15.3:0/3169193341 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc00e070 con 0x7fa8e415a770 2023-03-03T06:26:33.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.442+0000 7fa8e1ffb700 1 -- 172.21.15.3:0/3169193341 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8dc00bb20 con 0x7fa8e415a770 2023-03-03T06:26:33.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.442+0000 7fa8e1ffb700 1 -- 172.21.15.3:0/3169193341 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc012430 con 0x7fa8e415a770 2023-03-03T06:26:33.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.442+0000 7fa8e9701700 1 -- 172.21.15.3:0/3169193341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e415a770 msgr2=0x7fa8e415abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.442+0000 7fa8e9701700 1 --2- 172.21.15.3:0/3169193341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e415a770 0x7fa8e415abf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4061640 tx=0x7fa8dc00b330 comp rx=0 tx=0).stop 2023-03-03T06:26:33.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.442+0000 7fa8e9701700 1 -- 172.21.15.3:0/3169193341 shutdown_connections 2023-03-03T06:26:33.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.442+0000 7fa8e9701700 1 --2- 172.21.15.3:0/3169193341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e415a770 0x7fa8e415abf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.442+0000 7fa8e9701700 1 --2- 172.21.15.3:0/3169193341 >> v2:172.21.15.32:3300/0 conn(0x7fa8e4070820 0x7fa8e406b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.442+0000 7fa8e9701700 1 -- 172.21.15.3:0/3169193341 >> 172.21.15.3:0/3169193341 conn(0x7fa8e40ccc40 msgr2=0x7fa8e40cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:33.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.442+0000 7fa8e9701700 1 -- 172.21.15.3:0/3169193341 shutdown_connections 2023-03-03T06:26:33.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.442+0000 7fa8e9701700 1 -- 172.21.15.3:0/3169193341 wait complete. 2023-03-03T06:26:33.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e9701700 1 Processor -- start 2023-03-03T06:26:33.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e9701700 1 -- start start 2023-03-03T06:26:33.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e9701700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e406b580 0x7fa8e4155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:33.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e9701700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8e4070820 0x7fa8e40c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:33.451 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e9701700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8e415a770 0x7fa8e40c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:33.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e9701700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e415dc20 con 0x7fa8e406b580 2023-03-03T06:26:33.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e9701700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e415d660 con 0x7fa8e415a770 2023-03-03T06:26:33.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e9701700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e415d940 con 0x7fa8e4070820 2023-03-03T06:26:33.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e406b580 0x7fa8e4155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e406b580 0x7fa8e4155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41970/0 (socket says 172.21.15.3:41970) 2023-03-03T06:26:33.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e2ffd700 1 -- 172.21.15.3:0/3129731814 learned_addr learned my addr 172.21.15.3:0/3129731814 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:33.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e27fc700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8e4070820 0x7fa8e40c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e3fff700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8e415a770 0x7fa8e40c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e2ffd700 1 -- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8e4070820 msgr2=0x7fa8e40c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e2ffd700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8e4070820 0x7fa8e40c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e2ffd700 1 -- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8e415a770 msgr2=0x7fa8e40c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e2ffd700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8e415a770 0x7fa8e40c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e2ffd700 1 -- 172.21.15.3:0/3129731814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc009a70 con 0x7fa8e406b580 2023-03-03T06:26:33.454 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.446+0000 7fa8e2ffd700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e406b580 0x7fa8e4155f90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40010e0 tx=0x7fa8d400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:33.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.450+0000 7fa8cb7fe700 1 -- 172.21.15.3:0/3129731814 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40074e0 con 0x7fa8e406b580 2023-03-03T06:26:33.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.450+0000 7fa8cb7fe700 1 -- 172.21.15.3:0/3129731814 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d4007640 con 0x7fa8e406b580 2023-03-03T06:26:33.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.450+0000 7fa8e9701700 1 -- 172.21.15.3:0/3129731814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e415de90 con 0x7fa8e406b580 2023-03-03T06:26:33.455 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.450+0000 7fa8cb7fe700 1 -- 172.21.15.3:0/3129731814 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4005a70 con 0x7fa8e406b580 2023-03-03T06:26:33.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.450+0000 7fa8e9701700 1 -- 172.21.15.3:0/3129731814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e406be90 con 0x7fa8e406b580 2023-03-03T06:26:33.456 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.450+0000 7fa8e9701700 1 -- 172.21.15.3:0/3129731814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e415e170 con 0x7fa8e406b580 2023-03-03T06:26:33.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.454+0000 7fa8cb7fe700 1 -- 172.21.15.3:0/3129731814 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa8d4019030 con 0x7fa8e406b580 2023-03-03T06:26:33.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.454+0000 7fa8cb7fe700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa8d0064ee0 0x7fa8d0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:33.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.454+0000 7fa8cb7fe700 1 -- 172.21.15.3:0/3129731814 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa8d40815c0 con 0x7fa8e406b580 2023-03-03T06:26:33.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.454+0000 7fa8e27fc700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa8d0064ee0 0x7fa8d0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.461 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.458+0000 7fa8e27fc700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa8d0064ee0 0x7fa8d0067390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40b6110 tx=0x7fa8d8009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:33.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.462+0000 7fa8cb7fe700 1 -- 172.21.15.3:0/3129731814 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa8d404a400 con 0x7fa8e406b580 2023-03-03T06:26:33.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.642+0000 7fa8e9701700 1 -- 172.21.15.3:0/3129731814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8e404e180 con 0x7fa8e406b580 2023-03-03T06:26:33.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.642+0000 7fa8cb7fe700 1 -- 172.21.15.3:0/3129731814 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa8d404f370 con 0x7fa8e406b580 2023-03-03T06:26:33.649 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:33.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.646+0000 7fa8e9701700 1 -- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa8d0064ee0 msgr2=0x7fa8d0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.646+0000 7fa8e9701700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa8d0064ee0 0x7fa8d0067390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40b6110 tx=0x7fa8d8009240 comp rx=0 tx=0).stop 2023-03-03T06:26:33.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.646+0000 7fa8e9701700 1 -- 172.21.15.3:0/3129731814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e406b580 msgr2=0x7fa8e4155f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.646+0000 7fa8e9701700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e406b580 0x7fa8e4155f90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40010e0 tx=0x7fa8d400ca30 comp rx=0 tx=0).stop 2023-03-03T06:26:33.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.646+0000 7fa8e9701700 1 -- 172.21.15.3:0/3129731814 shutdown_connections 2023-03-03T06:26:33.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.646+0000 7fa8e9701700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa8d0064ee0 0x7fa8d0067390 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.646+0000 7fa8e9701700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8e415a770 0x7fa8e40c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.646+0000 7fa8e9701700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8e4070820 0x7fa8e40c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.646+0000 7fa8e9701700 1 --2- 172.21.15.3:0/3129731814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8e406b580 0x7fa8e4155f90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.646+0000 7fa8e9701700 1 -- 172.21.15.3:0/3129731814 >> 172.21.15.3:0/3129731814 conn(0x7fa8e40ccc40 msgr2=0x7fa8e4159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.646+0000 7fa8e9701700 1 -- 172.21.15.3:0/3129731814 shutdown_connections 2023-03-03T06:26:33.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.646+0000 7fa8e9701700 1 -- 172.21.15.3:0/3129731814 wait complete. 2023-03-03T06:26:33.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-03T06:26:33.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.886+0000 7fe85a558700 1 Processor -- start 2023-03-03T06:26:33.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.886+0000 7fe85a558700 1 -- start start 2023-03-03T06:26:33.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe85a558700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe854156ba0 0x7fe854071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:33.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe85a558700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe85415d6e0 0x7fe8540cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe85a558700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8540cf3b0 con 0x7fe8540cfd80 2023-03-03T06:26:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe85a558700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe854156f70 con 0x7fe854156ba0 2023-03-03T06:26:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe85a558700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8540cd320 con 0x7fe85415d6e0 2023-03-03T06:26:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe859d57700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe8540cfd80 0x7fe8540d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43120/0 (socket says 172.21.15.3:43120) 2023-03-03T06:26:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe859d57700 1 -- 172.21.15.3:0/468582028 learned_addr learned my addr 172.21.15.3:0/468582028 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:33.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe859556700 1 --2- 172.21.15.3:0/468582028 >> v2:172.21.15.3:3300/0 conn(0x7fe854156ba0 0x7fe854071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe858d55700 1 --2- 172.21.15.3:0/468582028 >> v2:172.21.15.32:3300/0 conn(0x7fe85415d6e0 0x7fe8540cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe84bfff700 1 -- 172.21.15.3:0/468582028 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804557457 0 0) 0x7fe8540cf3b0 con 0x7fe8540cfd80 2023-03-03T06:26:33.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe84bfff700 1 -- 172.21.15.3:0/468582028 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8400035d0 con 0x7fe8540cfd80 2023-03-03T06:26:33.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe859556700 1 -- 172.21.15.3:0/468582028 >> v2:172.21.15.32:3300/0 conn(0x7fe85415d6e0 msgr2=0x7fe8540cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:33.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe859556700 1 --2- 172.21.15.3:0/468582028 >> v2:172.21.15.32:3300/0 conn(0x7fe85415d6e0 0x7fe8540cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe859556700 1 -- 172.21.15.3:0/468582028 >> v1:172.21.15.3:6789/0 conn(0x7fe8540cfd80 legacy=0x7fe8540d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.890+0000 7fe859556700 1 -- 172.21.15.3:0/468582028 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8540cd600 con 0x7fe854156ba0 2023-03-03T06:26:33.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.894+0000 7fe859556700 1 --2- 172.21.15.3:0/468582028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe854156ba0 0x7fe854071280 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8540b1ca0 tx=0x7fe850009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f06f52a0792691 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:33.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.894+0000 7fe84bfff700 1 -- 172.21.15.3:0/468582028 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850011830 con 0x7fe854156ba0 2023-03-03T06:26:33.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.894+0000 7fe84bfff700 1 -- 172.21.15.3:0/468582028 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe850011990 con 0x7fe854156ba0 2023-03-03T06:26:33.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.894+0000 7fe84bfff700 1 -- 172.21.15.3:0/468582028 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850011d20 con 0x7fe854156ba0 2023-03-03T06:26:33.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.894+0000 7fe85a558700 1 -- 172.21.15.3:0/468582028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe854156ba0 msgr2=0x7fe854071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.894+0000 7fe85a558700 1 --2- 172.21.15.3:0/468582028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe854156ba0 0x7fe854071280 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8540b1ca0 tx=0x7fe850009b30 comp rx=0 tx=0).stop 2023-03-03T06:26:33.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.894+0000 7fe85a558700 1 -- 172.21.15.3:0/468582028 shutdown_connections 2023-03-03T06:26:33.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.894+0000 7fe85a558700 1 --2- 172.21.15.3:0/468582028 >> v2:172.21.15.32:3300/0 conn(0x7fe85415d6e0 0x7fe8540cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.894+0000 7fe85a558700 1 --2- 172.21.15.3:0/468582028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe854156ba0 0x7fe854071280 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.894+0000 7fe85a558700 1 -- 172.21.15.3:0/468582028 >> 172.21.15.3:0/468582028 conn(0x7fe85406e5c0 msgr2=0x7fe85406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:33.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.894+0000 7fe85a558700 1 -- 172.21.15.3:0/468582028 shutdown_connections 2023-03-03T06:26:33.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.894+0000 7fe85a558700 1 -- 172.21.15.3:0/468582028 wait complete. 2023-03-03T06:26:33.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.898+0000 7fe85a558700 1 Processor -- start 2023-03-03T06:26:33.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.898+0000 7fe85a558700 1 -- start start 2023-03-03T06:26:33.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe85a558700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8540cfd80 0x7fe8541599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:33.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe85a558700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe854156ba0 0x7fe8541f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:33.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe85a558700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe85415d6e0 0x7fe8540d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:33.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe85a558700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540c6870 con 0x7fe854156ba0 2023-03-03T06:26:33.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe85a558700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540b5a60 con 0x7fe85415d6e0 2023-03-03T06:26:33.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe85a558700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe854157660 con 0x7fe8540cfd80 2023-03-03T06:26:33.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe858d55700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe854156ba0 0x7fe8541f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe858d55700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe854156ba0 0x7fe8541f15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41994/0 (socket says 172.21.15.3:41994) 2023-03-03T06:26:33.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe858d55700 1 -- 172.21.15.3:0/947672045 learned_addr learned my addr 172.21.15.3:0/947672045 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:33.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe859556700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8540cfd80 0x7fe8541599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe859d57700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe85415d6e0 0x7fe8540d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe858d55700 1 -- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8540cfd80 msgr2=0x7fe8541599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe858d55700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8540cfd80 0x7fe8541599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe858d55700 1 -- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe85415d6e0 msgr2=0x7fe8540d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:33.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe858d55700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe85415d6e0 0x7fe8540d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:33.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe858d55700 1 -- 172.21.15.3:0/947672045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8500094b0 con 0x7fe854156ba0 2023-03-03T06:26:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe858d55700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe854156ba0 0x7fe8541f15a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe84400e9d0 tx=0x7fe84400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe849ffb700 1 -- 172.21.15.3:0/947672045 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844009720 con 0x7fe854156ba0 2023-03-03T06:26:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe849ffb700 1 -- 172.21.15.3:0/947672045 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84400ee70 con 0x7fe854156ba0 2023-03-03T06:26:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe85a558700 1 -- 172.21.15.3:0/947672045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8540717c0 con 0x7fe854156ba0 2023-03-03T06:26:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.902+0000 7fe849ffb700 1 -- 172.21.15.3:0/947672045 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84400ca50 con 0x7fe854156ba0 2023-03-03T06:26:33.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.906+0000 7fe85a558700 1 -- 172.21.15.3:0/947672045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8540cd8e0 con 0x7fe854156ba0 2023-03-03T06:26:33.912 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.906+0000 7fe85a558700 1 -- 172.21.15.3:0/947672045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8540cef40 con 0x7fe854156ba0 2023-03-03T06:26:33.914 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.910+0000 7fe849ffb700 1 -- 172.21.15.3:0/947672045 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe844019030 con 0x7fe854156ba0 2023-03-03T06:26:33.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.910+0000 7fe849ffb700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe838064ee0 0x7fe838067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:33.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.910+0000 7fe849ffb700 1 -- 172.21.15.3:0/947672045 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe844082270 con 0x7fe854156ba0 2023-03-03T06:26:33.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.914+0000 7fe859556700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe838064ee0 0x7fe838067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:33.919 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.914+0000 7fe859556700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe838064ee0 0x7fe838067390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe850000c00 tx=0x7fe850008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:33.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:33.918+0000 7fe849ffb700 1 -- 172.21.15.3:0/947672045 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe84404ff70 con 0x7fe854156ba0 2023-03-03T06:26:34.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.090+0000 7fe85a558700 1 -- 172.21.15.3:0/947672045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe854073e90 con 0x7fe854156ba0 2023-03-03T06:26:34.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.090+0000 7fe849ffb700 1 -- 172.21.15.3:0/947672045 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe844053590 con 0x7fe854156ba0 2023-03-03T06:26:34.098 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2023-03-03T06:26:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.094+0000 7fe85a558700 1 -- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe838064ee0 msgr2=0x7fe838067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.094+0000 7fe85a558700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe838064ee0 0x7fe838067390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe850000c00 tx=0x7fe850008030 comp rx=0 tx=0).stop 2023-03-03T06:26:34.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.094+0000 7fe85a558700 1 -- 172.21.15.3:0/947672045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe854156ba0 msgr2=0x7fe8541f15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.094+0000 7fe85a558700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe854156ba0 0x7fe8541f15a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe84400e9d0 tx=0x7fe84400ece0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.094+0000 7fe85a558700 1 -- 172.21.15.3:0/947672045 shutdown_connections 2023-03-03T06:26:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.094+0000 7fe85a558700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe838064ee0 0x7fe838067390 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.094+0000 7fe85a558700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe85415d6e0 0x7fe8540d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.094+0000 7fe85a558700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe854156ba0 0x7fe8541f15a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.094+0000 7fe85a558700 1 --2- 172.21.15.3:0/947672045 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe8540cfd80 0x7fe8541599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.094+0000 7fe85a558700 1 -- 172.21.15.3:0/947672045 >> 172.21.15.3:0/947672045 conn(0x7fe85406e5c0 msgr2=0x7fe854155d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.094+0000 7fe85a558700 1 -- 172.21.15.3:0/947672045 shutdown_connections 2023-03-03T06:26:34.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.094+0000 7fe85a558700 1 -- 172.21.15.3:0/947672045 wait complete. 2023-03-03T06:26:34.133 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-03T06:26:34.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.338+0000 7f0e631a1700 1 Processor -- start 2023-03-03T06:26:34.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.338+0000 7f0e631a1700 1 -- start start 2023-03-03T06:26:34.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.338+0000 7f0e631a1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0b07c0 0x7f0e5c0b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:34.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.338+0000 7f0e631a1700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e5c06c480 0x7f0e5c06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.338+0000 7f0e631a1700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5c0cdde0 con 0x7f0e5c1591d0 2023-03-03T06:26:34.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.342+0000 7f0e631a1700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5c1595a0 con 0x7f0e5c06c480 2023-03-03T06:26:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.342+0000 7f0e631a1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c0b1150 con 0x7f0e5c0b07c0 2023-03-03T06:26:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.342+0000 7f0e6219f700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0e5c1591d0 0x7f0e5c0cd6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43122/0 (socket says 172.21.15.3:43122) 2023-03-03T06:26:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.342+0000 7f0e6219f700 1 -- 172.21.15.3:0/132113177 learned_addr learned my addr 172.21.15.3:0/132113177 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.342+0000 7f0e6199e700 1 --2- 172.21.15.3:0/132113177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0b07c0 0x7f0e5c0b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:34.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.342+0000 7f0e629a0700 1 --2- 172.21.15.3:0/132113177 >> v2:172.21.15.32:3300/0 conn(0x7f0e5c06c480 0x7f0e5c06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:34.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.342+0000 7f0e6119d700 1 -- 172.21.15.3:0/132113177 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514676631 0 0) 0x7f0e5c0cdde0 con 0x7f0e5c1591d0 2023-03-03T06:26:34.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.342+0000 7f0e6199e700 1 -- 172.21.15.3:0/132113177 >> v2:172.21.15.32:3300/0 conn(0x7f0e5c06c480 msgr2=0x7f0e5c06c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:34.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.342+0000 7f0e6199e700 1 --2- 172.21.15.3:0/132113177 >> v2:172.21.15.32:3300/0 conn(0x7f0e5c06c480 0x7f0e5c06c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.342+0000 7f0e6199e700 1 -- 172.21.15.3:0/132113177 >> v1:172.21.15.3:6789/0 conn(0x7f0e5c1591d0 legacy=0x7f0e5c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:34.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.342+0000 7f0e6199e700 1 -- 172.21.15.3:0/132113177 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c0ce810 con 0x7f0e5c0b07c0 2023-03-03T06:26:34.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.342+0000 7f0e6199e700 1 --2- 172.21.15.3:0/132113177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0b07c0 0x7f0e5c0b0c10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009850 tx=0x7f0e4c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c15652c88f53de43 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:34.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.342+0000 7f0e6119d700 1 -- 172.21.15.3:0/132113177 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c011830 con 0x7f0e5c0b07c0 2023-03-03T06:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.346+0000 7f0e6119d700 1 -- 172.21.15.3:0/132113177 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4c011990 con 0x7f0e5c0b07c0 2023-03-03T06:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.346+0000 7f0e6119d700 1 -- 172.21.15.3:0/132113177 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c011d40 con 0x7f0e5c0b07c0 2023-03-03T06:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.346+0000 7f0e631a1700 1 -- 172.21.15.3:0/132113177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0b07c0 msgr2=0x7f0e5c0b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.346+0000 7f0e631a1700 1 --2- 172.21.15.3:0/132113177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0b07c0 0x7f0e5c0b0c10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009850 tx=0x7f0e4c009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.346+0000 7f0e631a1700 1 -- 172.21.15.3:0/132113177 shutdown_connections 2023-03-03T06:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.346+0000 7f0e631a1700 1 --2- 172.21.15.3:0/132113177 >> v2:172.21.15.32:3300/0 conn(0x7f0e5c06c480 0x7f0e5c06c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.346+0000 7f0e631a1700 1 --2- 172.21.15.3:0/132113177 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0b07c0 0x7f0e5c0b0c10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.346+0000 7f0e631a1700 1 -- 172.21.15.3:0/132113177 >> 172.21.15.3:0/132113177 conn(0x7f0e5c0607c0 msgr2=0x7f0e5c060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.346+0000 7f0e631a1700 1 -- 172.21.15.3:0/132113177 shutdown_connections 2023-03-03T06:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.346+0000 7f0e631a1700 1 -- 172.21.15.3:0/132113177 wait complete. 2023-03-03T06:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e631a1700 1 Processor -- start 2023-03-03T06:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e631a1700 1 -- start start 2023-03-03T06:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e631a1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5c06c480 0x7f0e5c14fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e631a1700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0b07c0 0x7f0e5c14c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e631a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e5c1591d0 0x7f0e5c0ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e631a1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c14f200 con 0x7f0e5c06c480 2023-03-03T06:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e631a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c14e5f0 con 0x7f0e5c1591d0 2023-03-03T06:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e631a1700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c14e8d0 con 0x7f0e5c0b07c0 2023-03-03T06:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e6219f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5c06c480 0x7f0e5c14fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e6219f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5c06c480 0x7f0e5c14fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42008/0 (socket says 172.21.15.3:42008) 2023-03-03T06:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e6219f700 1 -- 172.21.15.3:0/4070056702 learned_addr learned my addr 172.21.15.3:0/4070056702 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e6199e700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0b07c0 0x7f0e5c14c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e629a0700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e5c1591d0 0x7f0e5c0ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e6219f700 1 -- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0b07c0 msgr2=0x7f0e5c14c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e6219f700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0b07c0 0x7f0e5c14c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e6219f700 1 -- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e5c1591d0 msgr2=0x7f0e5c0ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e6219f700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e5c1591d0 0x7f0e5c0ac910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e6219f700 1 -- 172.21.15.3:0/4070056702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c0094b0 con 0x7f0e5c06c480 2023-03-03T06:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.350+0000 7f0e6219f700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5c06c480 0x7f0e5c14fe20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800cc20 tx=0x7f0e5800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.354+0000 7f0e52ffd700 1 -- 172.21.15.3:0/4070056702 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800ac20 con 0x7f0e5c06c480 2023-03-03T06:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.354+0000 7f0e52ffd700 1 -- 172.21.15.3:0/4070056702 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5800ad80 con 0x7f0e5c06c480 2023-03-03T06:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.354+0000 7f0e52ffd700 1 -- 172.21.15.3:0/4070056702 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58007310 con 0x7f0e5c06c480 2023-03-03T06:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.354+0000 7f0e631a1700 1 -- 172.21.15.3:0/4070056702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5c0cedd0 con 0x7f0e5c06c480 2023-03-03T06:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.354+0000 7f0e631a1700 1 -- 172.21.15.3:0/4070056702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5c0ceaf0 con 0x7f0e5c06c480 2023-03-03T06:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.354+0000 7f0e631a1700 1 -- 172.21.15.3:0/4070056702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5c0cf0b0 con 0x7f0e5c06c480 2023-03-03T06:26:34.362 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.358+0000 7f0e52ffd700 1 -- 172.21.15.3:0/4070056702 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f0e58010070 con 0x7f0e5c06c480 2023-03-03T06:26:34.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.362+0000 7f0e52ffd700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0e40064ee0 0x7f0e40067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:34.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.362+0000 7f0e52ffd700 1 -- 172.21.15.3:0/4070056702 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0e58081c80 con 0x7f0e5c06c480 2023-03-03T06:26:34.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.362+0000 7f0e6199e700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0e40064ee0 0x7f0e40067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.362+0000 7f0e6199e700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0e40064ee0 0x7f0e40067390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0090b0 tx=0x7f0e4c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.366+0000 7f0e52ffd700 1 -- 172.21.15.3:0/4070056702 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f0e5804fa00 con 0x7f0e5c06c480 2023-03-03T06:26:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.542+0000 7f0e631a1700 1 -- 172.21.15.3:0/4070056702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0e5c073e90 con 0x7f0e5c06c480 2023-03-03T06:26:34.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.546+0000 7f0e52ffd700 1 -- 172.21.15.3:0/4070056702 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0e580180a0 con 0x7f0e5c06c480 2023-03-03T06:26:34.550 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2023-03-03T06:26:34.553 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.546+0000 7f0e631a1700 1 -- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0e40064ee0 msgr2=0x7f0e40067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.546+0000 7f0e631a1700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0e40064ee0 0x7f0e40067390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0090b0 tx=0x7f0e4c008030 comp rx=0 tx=0).stop 2023-03-03T06:26:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.546+0000 7f0e631a1700 1 -- 172.21.15.3:0/4070056702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5c06c480 msgr2=0x7f0e5c14fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.546+0000 7f0e631a1700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5c06c480 0x7f0e5c14fe20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800cc20 tx=0x7f0e5800a060 comp rx=0 tx=0).stop 2023-03-03T06:26:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.546+0000 7f0e631a1700 1 -- 172.21.15.3:0/4070056702 shutdown_connections 2023-03-03T06:26:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.546+0000 7f0e631a1700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0e40064ee0 0x7f0e40067390 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.546+0000 7f0e631a1700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e5c1591d0 0x7f0e5c0ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.546+0000 7f0e631a1700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0e5c0b07c0 0x7f0e5c14c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.546+0000 7f0e631a1700 1 --2- 172.21.15.3:0/4070056702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e5c06c480 0x7f0e5c14fe20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.546+0000 7f0e631a1700 1 -- 172.21.15.3:0/4070056702 >> 172.21.15.3:0/4070056702 conn(0x7f0e5c0607c0 msgr2=0x7f0e5c14dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.546+0000 7f0e631a1700 1 -- 172.21.15.3:0/4070056702 shutdown_connections 2023-03-03T06:26:34.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.546+0000 7f0e631a1700 1 -- 172.21.15.3:0/4070056702 wait complete. 2023-03-03T06:26:34.585 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-03T06:26:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.782+0000 7f64f1441700 1 Processor -- start 2023-03-03T06:26:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64f1441700 1 -- start start 2023-03-03T06:26:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64f1441700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec1548c0 0x7f64ec0cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:34.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64f1441700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64ec0cc6d0 0x7f64ec0cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:34.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64f1441700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f64ec0cb780 0x7f64ec15a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:34.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64f1441700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec0cbba0 con 0x7f64ec0cb780 2023-03-03T06:26:34.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64f1441700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec0ca830 con 0x7f64ec0cc6d0 2023-03-03T06:26:34.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64f1441700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec154c90 con 0x7f64ec1548c0 2023-03-03T06:26:34.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64ebfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f64ec0cb780 0x7f64ec15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:34.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64ebfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f64ec0cb780 0x7f64ec15a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42016/0 (socket says 172.21.15.3:42016) 2023-03-03T06:26:34.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64ebfff700 1 -- 172.21.15.3:0/3381976051 learned_addr learned my addr 172.21.15.3:0/3381976051 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:34.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64eaffd700 1 --2- 172.21.15.3:0/3381976051 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec1548c0 0x7f64ec0cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:34.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64ea7fc700 1 --2- 172.21.15.3:0/3381976051 >> v2:172.21.15.32:3300/0 conn(0x7f64ec0cc6d0 0x7f64ec0cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64ebfff700 1 -- 172.21.15.3:0/3381976051 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec1548c0 msgr2=0x7f64ec0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64ebfff700 1 --2- 172.21.15.3:0/3381976051 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec1548c0 0x7f64ec0cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64ebfff700 1 -- 172.21.15.3:0/3381976051 >> v2:172.21.15.32:3300/0 conn(0x7f64ec0cc6d0 msgr2=0x7f64ec0cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64ebfff700 1 --2- 172.21.15.3:0/3381976051 >> v2:172.21.15.32:3300/0 conn(0x7f64ec0cc6d0 0x7f64ec0cb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.786+0000 7f64ebfff700 1 -- 172.21.15.3:0/3381976051 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec15b2b0 con 0x7f64ec0cb780 2023-03-03T06:26:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.790+0000 7f64ebfff700 1 --2- 172.21.15.3:0/3381976051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec0cb780 0x7f64ec15a730 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0cdd60 tx=0x7f64e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29149c296c131ca server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.790+0000 7f64e9ffb700 1 -- 172.21.15.3:0/3381976051 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400e070 con 0x7f64ec0cb780 2023-03-03T06:26:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.790+0000 7f64e9ffb700 1 -- 172.21.15.3:0/3381976051 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e400b910 con 0x7f64ec0cb780 2023-03-03T06:26:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.790+0000 7f64e9ffb700 1 -- 172.21.15.3:0/3381976051 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400bcc0 con 0x7f64ec0cb780 2023-03-03T06:26:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.790+0000 7f64f1441700 1 -- 172.21.15.3:0/3381976051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec0cb780 msgr2=0x7f64ec15a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.790+0000 7f64f1441700 1 --2- 172.21.15.3:0/3381976051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec0cb780 0x7f64ec15a730 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0cdd60 tx=0x7f64e400b330 comp rx=0 tx=0).stop 2023-03-03T06:26:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.790+0000 7f64f1441700 1 -- 172.21.15.3:0/3381976051 shutdown_connections 2023-03-03T06:26:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.790+0000 7f64f1441700 1 --2- 172.21.15.3:0/3381976051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec0cb780 0x7f64ec15a730 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.790+0000 7f64f1441700 1 --2- 172.21.15.3:0/3381976051 >> v2:172.21.15.32:3300/0 conn(0x7f64ec0cc6d0 0x7f64ec0cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.790+0000 7f64f1441700 1 --2- 172.21.15.3:0/3381976051 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec1548c0 0x7f64ec0cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.790+0000 7f64f1441700 1 -- 172.21.15.3:0/3381976051 >> 172.21.15.3:0/3381976051 conn(0x7f64ec06b580 msgr2=0x7f64ec06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.790+0000 7f64f1441700 1 -- 172.21.15.3:0/3381976051 shutdown_connections 2023-03-03T06:26:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.790+0000 7f64f1441700 1 -- 172.21.15.3:0/3381976051 wait complete. 2023-03-03T06:26:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.794+0000 7f64f1441700 1 Processor -- start 2023-03-03T06:26:34.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.794+0000 7f64f1441700 1 -- start start 2023-03-03T06:26:34.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.794+0000 7f64f1441700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec0cb780 0x7f64ec149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:34.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.794+0000 7f64f1441700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec0cc6d0 0x7f64ec147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:34.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.794+0000 7f64f1441700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec1548c0 0x7f64ec144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:34.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.794+0000 7f64f1441700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec15d570 con 0x7f64ec0cb780 2023-03-03T06:26:34.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.794+0000 7f64f1441700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec152cc0 con 0x7f64ec1548c0 2023-03-03T06:26:34.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.794+0000 7f64f1441700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0b3010 con 0x7f64ec0cc6d0 2023-03-03T06:26:34.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.794+0000 7f64eaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec0cb780 0x7f64ec149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:34.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.794+0000 7f64eaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec0cb780 0x7f64ec149c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42030/0 (socket says 172.21.15.3:42030) 2023-03-03T06:26:34.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.794+0000 7f64eaffd700 1 -- 172.21.15.3:0/1298105044 learned_addr learned my addr 172.21.15.3:0/1298105044 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:34.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.794+0000 7f64ebfff700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec1548c0 0x7f64ec144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:34.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.794+0000 7f64ea7fc700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec0cc6d0 0x7f64ec147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:34.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.798+0000 7f64eaffd700 1 -- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec0cc6d0 msgr2=0x7f64ec147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:34.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.798+0000 7f64eaffd700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec0cc6d0 0x7f64ec147020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.798+0000 7f64eaffd700 1 -- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec1548c0 msgr2=0x7f64ec144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:34.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.798+0000 7f64eaffd700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec1548c0 0x7f64ec144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:34.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.798+0000 7f64eaffd700 1 -- 172.21.15.3:0/1298105044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e4009b30 con 0x7f64ec0cb780 2023-03-03T06:26:34.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.798+0000 7f64eaffd700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec0cb780 0x7f64ec149c80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00ec10 tx=0x7f64dc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:34.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.798+0000 7f64d37fe700 1 -- 172.21.15.3:0/1298105044 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc009740 con 0x7f64ec0cb780 2023-03-03T06:26:34.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.798+0000 7f64d37fe700 1 -- 172.21.15.3:0/1298105044 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc00c850 con 0x7f64ec0cb780 2023-03-03T06:26:34.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.798+0000 7f64f1441700 1 -- 172.21.15.3:0/1298105044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec0caa40 con 0x7f64ec0cb780 2023-03-03T06:26:34.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.798+0000 7f64d37fe700 1 -- 172.21.15.3:0/1298105044 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc00cbe0 con 0x7f64ec0cb780 2023-03-03T06:26:34.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.798+0000 7f64f1441700 1 -- 172.21.15.3:0/1298105044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec15b590 con 0x7f64ec0cb780 2023-03-03T06:26:34.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.798+0000 7f64f1441700 1 -- 172.21.15.3:0/1298105044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec0cad20 con 0x7f64ec0cb780 2023-03-03T06:26:34.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.802+0000 7f64d37fe700 1 -- 172.21.15.3:0/1298105044 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f64dc019030 con 0x7f64ec0cb780 2023-03-03T06:26:34.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.802+0000 7f64d37fe700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f64d8064ee0 0x7f64d8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:34.812 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.802+0000 7f64d37fe700 1 -- 172.21.15.3:0/1298105044 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f64dc0821e0 con 0x7f64ec0cb780 2023-03-03T06:26:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.806+0000 7f64ea7fc700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f64d8064ee0 0x7f64d8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.806+0000 7f64ea7fc700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f64d8064ee0 0x7f64d8067390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f64ec149440 tx=0x7f64e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:34.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:34.810+0000 7f64d37fe700 1 -- 172.21.15.3:0/1298105044 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f64dc04fee0 con 0x7f64ec0cb780 2023-03-03T06:26:35.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.026+0000 7f64f1441700 1 -- 172.21.15.3:0/1298105044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64ec04e180 con 0x7f64ec0cb780 2023-03-03T06:26:35.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.026+0000 7f64d37fe700 1 -- 172.21.15.3:0/1298105044 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4635 (secure 0 0 0) 0x7f64dc053500 con 0x7f64ec0cb780 2023-03-03T06:26:35.033 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:35.033 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2744699972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2744699972},{"type":"v1","addr":"172.21.15.3:6834","nonce":2744699972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/708054272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":708054272},{"type":"v1","addr":"172.21.15.3:6835","nonce":708054272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2935923413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2935923413},{"type":"v1","addr":"172.21.15.32:6837","nonce":2935923413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-03T06:25:13.798672+0000","modified":"2023-03-03T06:25:41.755082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/691220172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":691220172},{"type":"v1","addr":"172.21.15.32:6836","nonce":691220172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-03-03T06:26:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.030+0000 7f64f1441700 1 -- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f64d8064ee0 msgr2=0x7f64d8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.030+0000 7f64f1441700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f64d8064ee0 0x7f64d8067390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f64ec149440 tx=0x7f64e0008040 comp rx=0 tx=0).stop 2023-03-03T06:26:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.030+0000 7f64f1441700 1 -- 172.21.15.3:0/1298105044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec0cb780 msgr2=0x7f64ec149c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.030+0000 7f64f1441700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec0cb780 0x7f64ec149c80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00ec10 tx=0x7f64dc00ef20 comp rx=0 tx=0).stop 2023-03-03T06:26:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.030+0000 7f64f1441700 1 -- 172.21.15.3:0/1298105044 shutdown_connections 2023-03-03T06:26:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.030+0000 7f64f1441700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f64d8064ee0 0x7f64d8067390 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.030+0000 7f64f1441700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64ec1548c0 0x7f64ec144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.030+0000 7f64f1441700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f64ec0cc6d0 0x7f64ec147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.030+0000 7f64f1441700 1 --2- 172.21.15.3:0/1298105044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f64ec0cb780 0x7f64ec149c80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.030+0000 7f64f1441700 1 -- 172.21.15.3:0/1298105044 >> 172.21.15.3:0/1298105044 conn(0x7f64ec06b580 msgr2=0x7f64ec15b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:35.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.030+0000 7f64f1441700 1 -- 172.21.15.3:0/1298105044 shutdown_connections 2023-03-03T06:26:35.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.030+0000 7f64f1441700 1 -- 172.21.15.3:0/1298105044 wait complete. 2023-03-03T06:26:35.040 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2023-03-03T06:26:35.070 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:35.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.270+0000 7f9fddbce700 1 Processor -- start 2023-03-03T06:26:35.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.270+0000 7f9fddbce700 1 -- start start 2023-03-03T06:26:35.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.274+0000 7f9fddbce700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fd81608d0 0x7f9fd80cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:35.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.274+0000 7f9fddbce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fd80cfb60 0x7f9fd80d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:35.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.274+0000 7f9fddbce700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd8160440 con 0x7f9fd8153670 2023-03-03T06:26:35.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.274+0000 7f9fddbce700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd80cffe0 con 0x7f9fd81608d0 2023-03-03T06:26:35.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.274+0000 7f9fddbce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80cd3e0 con 0x7f9fd80cfb60 2023-03-03T06:26:35.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.274+0000 7f9fdcbcc700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9fd8153670 0x7f9fd8071350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:58318/0 (socket says 172.21.15.3:58318) 2023-03-03T06:26:35.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.274+0000 7f9fdcbcc700 1 -- 172.21.15.3:0/3221999974 learned_addr learned my addr 172.21.15.3:0/3221999974 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:35.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.274+0000 7f9fdd3cd700 1 --2- 172.21.15.3:0/3221999974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fd80cfb60 0x7f9fd80d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:35.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.274+0000 7f9fcffff700 1 --2- 172.21.15.3:0/3221999974 >> v2:172.21.15.32:3300/0 conn(0x7f9fd81608d0 0x7f9fd80cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:35.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fdd3cd700 1 -- 172.21.15.3:0/3221999974 >> v2:172.21.15.32:3300/0 conn(0x7f9fd81608d0 msgr2=0x7f9fd80cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:35.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fdd3cd700 1 --2- 172.21.15.3:0/3221999974 >> v2:172.21.15.32:3300/0 conn(0x7f9fd81608d0 0x7f9fd80cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fdd3cd700 1 -- 172.21.15.3:0/3221999974 >> v1:172.21.15.32:6789/0 conn(0x7f9fd8153670 legacy=0x7f9fd8071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fdd3cd700 1 -- 172.21.15.3:0/3221999974 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd80cd6c0 con 0x7f9fd80cfb60 2023-03-03T06:26:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fdd3cd700 1 --2- 172.21.15.3:0/3221999974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fd80cfb60 0x7f9fd80d0b10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009880 tx=0x7f9fd0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d9f4b7313bbc74 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fcf7fe700 1 -- 172.21.15.3:0/3221999974 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0011830 con 0x7f9fd80cfb60 2023-03-03T06:26:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fcf7fe700 1 -- 172.21.15.3:0/3221999974 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd0011990 con 0x7f9fd80cfb60 2023-03-03T06:26:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fcf7fe700 1 -- 172.21.15.3:0/3221999974 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0011d20 con 0x7f9fd80cfb60 2023-03-03T06:26:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fddbce700 1 -- 172.21.15.3:0/3221999974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fd80cfb60 msgr2=0x7f9fd80d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fddbce700 1 --2- 172.21.15.3:0/3221999974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fd80cfb60 0x7f9fd80d0b10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009880 tx=0x7f9fd0009b90 comp rx=0 tx=0).stop 2023-03-03T06:26:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fddbce700 1 -- 172.21.15.3:0/3221999974 shutdown_connections 2023-03-03T06:26:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fddbce700 1 --2- 172.21.15.3:0/3221999974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fd80cfb60 0x7f9fd80d0b10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fddbce700 1 --2- 172.21.15.3:0/3221999974 >> v2:172.21.15.32:3300/0 conn(0x7f9fd81608d0 0x7f9fd80cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.278+0000 7f9fddbce700 1 -- 172.21.15.3:0/3221999974 >> 172.21.15.3:0/3221999974 conn(0x7f9fd806e5c0 msgr2=0x7f9fd806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.282+0000 7f9fddbce700 1 -- 172.21.15.3:0/3221999974 shutdown_connections 2023-03-03T06:26:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.282+0000 7f9fddbce700 1 -- 172.21.15.3:0/3221999974 wait complete. 2023-03-03T06:26:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.282+0000 7f9fddbce700 1 Processor -- start 2023-03-03T06:26:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.282+0000 7f9fddbce700 1 -- start start 2023-03-03T06:26:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fddbce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd80cfb60 0x7f9fd8152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fddbce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fd8153670 0x7f9fd80b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fddbce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd81608d0 0x7f9fd815df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fddbce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd8152aa0 con 0x7f9fd80cfb60 2023-03-03T06:26:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fddbce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80b4370 con 0x7f9fd81608d0 2023-03-03T06:26:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fddbce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80b4650 con 0x7f9fd8153670 2023-03-03T06:26:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fdcbcc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd80cfb60 0x7f9fd8152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fcffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fd8153670 0x7f9fd80b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fcffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fd8153670 0x7f9fd80b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:44090/0 (socket says 172.21.15.3:44090) 2023-03-03T06:26:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fcffff700 1 -- 172.21.15.3:0/332860266 learned_addr learned my addr 172.21.15.3:0/332860266 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fdcbcc700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd80cfb60 0x7f9fd8152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42036/0 (socket says 172.21.15.3:42036) 2023-03-03T06:26:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fdd3cd700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd81608d0 0x7f9fd815df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fdcbcc700 1 -- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fd8153670 msgr2=0x7f9fd80b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fdcbcc700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fd8153670 0x7f9fd80b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fdcbcc700 1 -- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd81608d0 msgr2=0x7f9fd815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fdcbcc700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd81608d0 0x7f9fd815df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fdcbcc700 1 -- 172.21.15.3:0/332860266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd00094b0 con 0x7f9fd80cfb60 2023-03-03T06:26:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fcffff700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fd8153670 0x7f9fd80b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.286+0000 7f9fdcbcc700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd80cfb60 0x7f9fd8152560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400cae0 tx=0x7f9fd400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.290+0000 7f9fcd7fa700 1 -- 172.21.15.3:0/332860266 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4007480 con 0x7f9fd80cfb60 2023-03-03T06:26:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.290+0000 7f9fcd7fa700 1 -- 172.21.15.3:0/332860266 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd40075e0 con 0x7f9fd80cfb60 2023-03-03T06:26:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.290+0000 7f9fcd7fa700 1 -- 172.21.15.3:0/332860266 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4007950 con 0x7f9fd80cfb60 2023-03-03T06:26:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.290+0000 7f9fddbce700 1 -- 172.21.15.3:0/332860266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd80cd9a0 con 0x7f9fd80cfb60 2023-03-03T06:26:35.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.290+0000 7f9fddbce700 1 -- 172.21.15.3:0/332860266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd81600d0 con 0x7f9fd80cfb60 2023-03-03T06:26:35.297 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.290+0000 7f9fddbce700 1 -- 172.21.15.3:0/332860266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd80cdb30 con 0x7f9fd80cfb60 2023-03-03T06:26:35.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.294+0000 7f9fcd7fa700 1 -- 172.21.15.3:0/332860266 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9fd4019030 con 0x7f9fd80cfb60 2023-03-03T06:26:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.294+0000 7f9fcd7fa700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9fc4064ee0 0x7f9fc4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.294+0000 7f9fcd7fa700 1 -- 172.21.15.3:0/332860266 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9fd40817e0 con 0x7f9fd80cfb60 2023-03-03T06:26:35.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.298+0000 7f9fcffff700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9fc4064ee0 0x7f9fc4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.298+0000 7f9fcffff700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9fc4064ee0 0x7f9fc4067390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80b2290 tx=0x7f9fc8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:35.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.302+0000 7f9fcd7fa700 1 -- 172.21.15.3:0/332860266 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9fd404a400 con 0x7f9fd80cfb60 2023-03-03T06:26:35.485 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.478+0000 7f9fddbce700 1 -- 172.21.15.3:0/332860266 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fd8073e90 con 0x7f9fd80cfb60 2023-03-03T06:26:35.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.482+0000 7f9fcd7fa700 1 -- 172.21.15.3:0/332860266 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f9fd404f700 con 0x7f9fd80cfb60 2023-03-03T06:26:35.489 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:35.489 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:53.877945+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:35.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.486+0000 7f9fddbce700 1 -- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9fc4064ee0 msgr2=0x7f9fc4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:35.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.486+0000 7f9fddbce700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9fc4064ee0 0x7f9fc4067390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80b2290 tx=0x7f9fc8008040 comp rx=0 tx=0).stop 2023-03-03T06:26:35.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.486+0000 7f9fddbce700 1 -- 172.21.15.3:0/332860266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd80cfb60 msgr2=0x7f9fd8152560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:35.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.486+0000 7f9fddbce700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd80cfb60 0x7f9fd8152560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400cae0 tx=0x7f9fd400cb10 comp rx=0 tx=0).stop 2023-03-03T06:26:35.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.486+0000 7f9fddbce700 1 -- 172.21.15.3:0/332860266 shutdown_connections 2023-03-03T06:26:35.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.486+0000 7f9fddbce700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9fc4064ee0 0x7f9fc4067390 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.486+0000 7f9fddbce700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd81608d0 0x7f9fd815df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.486+0000 7f9fddbce700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9fd8153670 0x7f9fd80b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.486+0000 7f9fddbce700 1 --2- 172.21.15.3:0/332860266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fd80cfb60 0x7f9fd8152560 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.486+0000 7f9fddbce700 1 -- 172.21.15.3:0/332860266 >> 172.21.15.3:0/332860266 conn(0x7f9fd806e5c0 msgr2=0x7f9fd80cbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:35.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.486+0000 7f9fddbce700 1 -- 172.21.15.3:0/332860266 shutdown_connections 2023-03-03T06:26:35.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.486+0000 7f9fddbce700 1 -- 172.21.15.3:0/332860266 wait complete. 2023-03-03T06:26:35.527 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-03-03T06:26:35.528 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-03T06:26:35.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.726+0000 7f02fc97b700 1 Processor -- start 2023-03-03T06:26:35.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.726+0000 7f02fc97b700 1 -- start start 2023-03-03T06:26:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.726+0000 7f02fc97b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f02f80cb990 0x7f02f80cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.726+0000 7f02fc97b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02f806db90 0x7f02f806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.726+0000 7f02fc97b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02f80b1c70 0x7f02f806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.726+0000 7f02fc97b700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f80cd410 con 0x7f02f80cb990 2023-03-03T06:26:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.726+0000 7f02fc97b700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f806e130 con 0x7f02f80b1c70 2023-03-03T06:26:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.726+0000 7f02fc97b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80b12c0 con 0x7f02f806db90 2023-03-03T06:26:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.726+0000 7f02f6d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f02f80cb990 0x7f02f80cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.726+0000 7f02f6d9d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f02f80cb990 0x7f02f80cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42042/0 (socket says 172.21.15.3:42042) 2023-03-03T06:26:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.726+0000 7f02f6d9d700 1 -- 172.21.15.3:0/3582406136 learned_addr learned my addr 172.21.15.3:0/3582406136 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.726+0000 7f02f659c700 1 --2- 172.21.15.3:0/3582406136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02f806db90 0x7f02f806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02f759e700 1 --2- 172.21.15.3:0/3582406136 >> v2:172.21.15.32:3300/0 conn(0x7f02f80b1c70 0x7f02f806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02f6d9d700 1 -- 172.21.15.3:0/3582406136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02f806db90 msgr2=0x7f02f806e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02f6d9d700 1 --2- 172.21.15.3:0/3582406136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02f806db90 0x7f02f806e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02f6d9d700 1 -- 172.21.15.3:0/3582406136 >> v2:172.21.15.32:3300/0 conn(0x7f02f80b1c70 msgr2=0x7f02f806eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02f6d9d700 1 --2- 172.21.15.3:0/3582406136 >> v2:172.21.15.32:3300/0 conn(0x7f02f80b1c70 0x7f02f806eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02f6d9d700 1 -- 172.21.15.3:0/3582406136 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f80b15a0 con 0x7f02f80cb990 2023-03-03T06:26:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02f6d9d700 1 --2- 172.21.15.3:0/3582406136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02f80cb990 0x7f02f80cd8a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f02f8160fc0 tx=0x7f02e8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac7884ef86f899c server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02f5d9b700 1 -- 172.21.15.3:0/3582406136 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e8011830 con 0x7f02f80cb990 2023-03-03T06:26:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02f5d9b700 1 -- 172.21.15.3:0/3582406136 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e8011990 con 0x7f02f80cb990 2023-03-03T06:26:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02f5d9b700 1 -- 172.21.15.3:0/3582406136 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e8011d20 con 0x7f02f80cb990 2023-03-03T06:26:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02fc97b700 1 -- 172.21.15.3:0/3582406136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02f80cb990 msgr2=0x7f02f80cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02fc97b700 1 --2- 172.21.15.3:0/3582406136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02f80cb990 0x7f02f80cd8a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f02f8160fc0 tx=0x7f02e8009bf0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02fc97b700 1 -- 172.21.15.3:0/3582406136 shutdown_connections 2023-03-03T06:26:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02fc97b700 1 --2- 172.21.15.3:0/3582406136 >> v2:172.21.15.32:3300/0 conn(0x7f02f80b1c70 0x7f02f806eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02fc97b700 1 --2- 172.21.15.3:0/3582406136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02f806db90 0x7f02f806e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02fc97b700 1 --2- 172.21.15.3:0/3582406136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02f80cb990 0x7f02f80cd8a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02fc97b700 1 -- 172.21.15.3:0/3582406136 >> 172.21.15.3:0/3582406136 conn(0x7f02f8069d70 msgr2=0x7f02f806a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02fc97b700 1 -- 172.21.15.3:0/3582406136 shutdown_connections 2023-03-03T06:26:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.730+0000 7f02fc97b700 1 -- 172.21.15.3:0/3582406136 wait complete. 2023-03-03T06:26:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.734+0000 7f02fc97b700 1 Processor -- start 2023-03-03T06:26:35.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.734+0000 7f02fc97b700 1 -- start start 2023-03-03T06:26:35.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02fc97b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02f806db90 0x7f02f8159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:35.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02fc97b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f80b1c70 0x7f02f80c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:35.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02fc97b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02f80cb990 0x7f02f81e7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:35.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02fc97b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80ad090 con 0x7f02f806db90 2023-03-03T06:26:35.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02fc97b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80ac130 con 0x7f02f80b1c70 2023-03-03T06:26:35.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02fc97b700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80ac410 con 0x7f02f80cb990 2023-03-03T06:26:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02f6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02f806db90 0x7f02f8159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02f6d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02f806db90 0x7f02f8159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42054/0 (socket says 172.21.15.3:42054) 2023-03-03T06:26:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02f6d9d700 1 -- 172.21.15.3:0/1199327007 learned_addr learned my addr 172.21.15.3:0/1199327007 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02f659c700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f80b1c70 0x7f02f80c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02f759e700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02f80cb990 0x7f02f81e7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02f6d9d700 1 -- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02f80cb990 msgr2=0x7f02f81e7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02f6d9d700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02f80cb990 0x7f02f81e7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02f6d9d700 1 -- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f80b1c70 msgr2=0x7f02f80c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02f6d9d700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f80b1c70 0x7f02f80c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02f6d9d700 1 -- 172.21.15.3:0/1199327007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e8009570 con 0x7f02f806db90 2023-03-03T06:26:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02f6d9d700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02f806db90 0x7f02f8159fa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f02f806b210 tx=0x7f02e8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02df7fe700 1 -- 172.21.15.3:0/1199327007 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e8004240 con 0x7f02f806db90 2023-03-03T06:26:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02df7fe700 1 -- 172.21.15.3:0/1199327007 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e801f070 con 0x7f02f806db90 2023-03-03T06:26:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02fc97b700 1 -- 172.21.15.3:0/1199327007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f80cd080 con 0x7f02f806db90 2023-03-03T06:26:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.738+0000 7f02df7fe700 1 -- 172.21.15.3:0/1199327007 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e80045e0 con 0x7f02f806db90 2023-03-03T06:26:35.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.742+0000 7f02fc97b700 1 -- 172.21.15.3:0/1199327007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f80b1880 con 0x7f02f806db90 2023-03-03T06:26:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.742+0000 7f02fc97b700 1 -- 172.21.15.3:0/1199327007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f80cd210 con 0x7f02f806db90 2023-03-03T06:26:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.746+0000 7f02df7fe700 1 -- 172.21.15.3:0/1199327007 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f02e800f050 con 0x7f02f806db90 2023-03-03T06:26:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.746+0000 7f02df7fe700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f02e4064ee0 0x7f02e4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.746+0000 7f02df7fe700 1 -- 172.21.15.3:0/1199327007 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f02e8083460 con 0x7f02f806db90 2023-03-03T06:26:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.746+0000 7f02f659c700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f02e4064ee0 0x7f02e4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.750+0000 7f02f659c700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f02e4064ee0 0x7f02e4067390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f02f81f5570 tx=0x7f02ec009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:35.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.750+0000 7f02df7fe700 1 -- 172.21.15.3:0/1199327007 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f02e80512a0 con 0x7f02f806db90 2023-03-03T06:26:35.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.962+0000 7f02fc97b700 1 -- 172.21.15.3:0/1199327007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02f8073e90 con 0x7f02f806db90 2023-03-03T06:26:35.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.962+0000 7f02df7fe700 1 -- 172.21.15.3:0/1199327007 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4635 (secure 0 0 0) 0x7f02e80548c0 con 0x7f02f806db90 2023-03-03T06:26:35.969 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:35.970 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2744699972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2744699972},{"type":"v1","addr":"172.21.15.3:6834","nonce":2744699972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/708054272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":708054272},{"type":"v1","addr":"172.21.15.3:6835","nonce":708054272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2935923413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2935923413},{"type":"v1","addr":"172.21.15.32:6837","nonce":2935923413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-03T06:25:13.798672+0000","modified":"2023-03-03T06:25:41.755082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/691220172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":691220172},{"type":"v1","addr":"172.21.15.32:6836","nonce":691220172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-03-03T06:26:35.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.966+0000 7f02fc97b700 1 -- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f02e4064ee0 msgr2=0x7f02e4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:35.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.966+0000 7f02fc97b700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f02e4064ee0 0x7f02e4067390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f02f81f5570 tx=0x7f02ec009240 comp rx=0 tx=0).stop 2023-03-03T06:26:35.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.966+0000 7f02fc97b700 1 -- 172.21.15.3:0/1199327007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02f806db90 msgr2=0x7f02f8159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:35.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.966+0000 7f02fc97b700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02f806db90 0x7f02f8159fa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f02f806b210 tx=0x7f02e8004770 comp rx=0 tx=0).stop 2023-03-03T06:26:35.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.966+0000 7f02fc97b700 1 -- 172.21.15.3:0/1199327007 shutdown_connections 2023-03-03T06:26:35.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.966+0000 7f02fc97b700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f02e4064ee0 0x7f02e4067390 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.966+0000 7f02fc97b700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f02f80cb990 0x7f02f81e7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.966+0000 7f02fc97b700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f80b1c70 0x7f02f80c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.966+0000 7f02fc97b700 1 --2- 172.21.15.3:0/1199327007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02f806db90 0x7f02f8159fa0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:35.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.966+0000 7f02fc97b700 1 -- 172.21.15.3:0/1199327007 >> 172.21.15.3:0/1199327007 conn(0x7f02f8069d70 msgr2=0x7f02f806e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:35.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.966+0000 7f02fc97b700 1 -- 172.21.15.3:0/1199327007 shutdown_connections 2023-03-03T06:26:35.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:35.966+0000 7f02fc97b700 1 -- 172.21.15.3:0/1199327007 wait complete. 2023-03-03T06:26:35.978 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2023-03-03T06:26:36.010 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-03T06:26:36.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.214+0000 7fd37d09f700 1 Processor -- start 2023-03-03T06:26:36.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.218+0000 7fd37d09f700 1 -- start start 2023-03-03T06:26:36.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd37d09f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3781608d0 0x7fd3780cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:36.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd37d09f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3780cfb60 0x7fd3780d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd37d09f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd378160440 con 0x7fd378153670 2023-03-03T06:26:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd37d09f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3780cffa0 con 0x7fd3781608d0 2023-03-03T06:26:36.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd37d09f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3780cd3e0 con 0x7fd3780cfb60 2023-03-03T06:26:36.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd3777fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3781608d0 0x7fd3780cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd377fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd378153670 0x7fd378071350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43136/0 (socket says 172.21.15.3:43136) 2023-03-03T06:26:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd377fff700 1 -- 172.21.15.3:0/405719006 learned_addr learned my addr 172.21.15.3:0/405719006 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:36.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd37c89e700 1 --2- 172.21.15.3:0/405719006 >> v2:172.21.15.32:3300/0 conn(0x7fd3780cfb60 0x7fd3780d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:36.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd3777fe700 1 -- 172.21.15.3:0/405719006 >> v2:172.21.15.32:3300/0 conn(0x7fd3780cfb60 msgr2=0x7fd3780d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:36.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd3777fe700 1 --2- 172.21.15.3:0/405719006 >> v2:172.21.15.32:3300/0 conn(0x7fd3780cfb60 0x7fd3780d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd3777fe700 1 -- 172.21.15.3:0/405719006 >> v1:172.21.15.3:6789/0 conn(0x7fd378153670 legacy=0x7fd378071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:36.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd3777fe700 1 -- 172.21.15.3:0/405719006 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780cd6c0 con 0x7fd3781608d0 2023-03-03T06:26:36.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd376ffd700 1 -- 172.21.15.3:0/405719006 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002136625 0 0) 0x7fd378160440 con 0x7fd378153670 2023-03-03T06:26:36.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.222+0000 7fd3777fe700 1 --2- 172.21.15.3:0/405719006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3781608d0 0x7fd3780cf620 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd36c009850 tx=0x7fd36c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf1cefd30d365a server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:36.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.226+0000 7fd376ffd700 1 -- 172.21.15.3:0/405719006 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c011830 con 0x7fd3781608d0 2023-03-03T06:26:36.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.226+0000 7fd376ffd700 1 -- 172.21.15.3:0/405719006 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c011990 con 0x7fd3781608d0 2023-03-03T06:26:36.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.226+0000 7fd376ffd700 1 -- 172.21.15.3:0/405719006 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c011d20 con 0x7fd3781608d0 2023-03-03T06:26:36.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.226+0000 7fd37d09f700 1 -- 172.21.15.3:0/405719006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3781608d0 msgr2=0x7fd3780cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:36.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.226+0000 7fd37d09f700 1 --2- 172.21.15.3:0/405719006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3781608d0 0x7fd3780cf620 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd36c009850 tx=0x7fd36c009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:36.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.226+0000 7fd37d09f700 1 -- 172.21.15.3:0/405719006 shutdown_connections 2023-03-03T06:26:36.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.226+0000 7fd37d09f700 1 --2- 172.21.15.3:0/405719006 >> v2:172.21.15.32:3300/0 conn(0x7fd3780cfb60 0x7fd3780d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.226+0000 7fd37d09f700 1 --2- 172.21.15.3:0/405719006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3781608d0 0x7fd3780cf620 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.226+0000 7fd37d09f700 1 -- 172.21.15.3:0/405719006 >> 172.21.15.3:0/405719006 conn(0x7fd37806e5c0 msgr2=0x7fd37806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:36.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.226+0000 7fd37d09f700 1 -- 172.21.15.3:0/405719006 shutdown_connections 2023-03-03T06:26:36.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.226+0000 7fd37d09f700 1 -- 172.21.15.3:0/405719006 wait complete. 2023-03-03T06:26:36.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.230+0000 7fd37d09f700 1 Processor -- start 2023-03-03T06:26:36.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.230+0000 7fd37d09f700 1 -- start start 2023-03-03T06:26:36.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.230+0000 7fd37d09f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3780cfb60 0x7fd37814cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:36.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.230+0000 7fd37d09f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd378153670 0x7fd3780ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:36.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.230+0000 7fd37d09f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3781608d0 0x7fd3780b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:36.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd37d09f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780cb690 con 0x7fd378153670 2023-03-03T06:26:36.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd37d09f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780b2780 con 0x7fd3780cfb60 2023-03-03T06:26:36.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd37d09f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780cb0d0 con 0x7fd3781608d0 2023-03-03T06:26:36.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd3777fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd378153670 0x7fd3780ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:36.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd3777fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd378153670 0x7fd3780ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42078/0 (socket says 172.21.15.3:42078) 2023-03-03T06:26:36.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd3777fe700 1 -- 172.21.15.3:0/1012005992 learned_addr learned my addr 172.21.15.3:0/1012005992 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:36.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd377fff700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3780cfb60 0x7fd37814cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:36.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd37c89e700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3781608d0 0x7fd3780b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:36.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd3777fe700 1 -- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3781608d0 msgr2=0x7fd3780b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:36.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd3777fe700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3781608d0 0x7fd3780b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd3777fe700 1 -- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3780cfb60 msgr2=0x7fd37814cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:36.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd3777fe700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3780cfb60 0x7fd37814cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd3777fe700 1 -- 172.21.15.3:0/1012005992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36c0094b0 con 0x7fd378153670 2023-03-03T06:26:36.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd377fff700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3780cfb60 0x7fd37814cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:36.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd3777fe700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd378153670 0x7fd3780ca6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd36c014040 tx=0x7fd36c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:36.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd374ff9700 1 -- 172.21.15.3:0/1012005992 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c0036c0 con 0x7fd378153670 2023-03-03T06:26:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd374ff9700 1 -- 172.21.15.3:0/1012005992 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c01e070 con 0x7fd378153670 2023-03-03T06:26:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd374ff9700 1 -- 172.21.15.3:0/1012005992 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c003a40 con 0x7fd378153670 2023-03-03T06:26:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd37d09f700 1 -- 172.21.15.3:0/1012005992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3780cd9a0 con 0x7fd378153670 2023-03-03T06:26:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.234+0000 7fd37d09f700 1 -- 172.21.15.3:0/1012005992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3781600d0 con 0x7fd378153670 2023-03-03T06:26:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.238+0000 7fd37d09f700 1 -- 172.21.15.3:0/1012005992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3780cdb30 con 0x7fd378153670 2023-03-03T06:26:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.238+0000 7fd374ff9700 1 -- 172.21.15.3:0/1012005992 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd36c00f050 con 0x7fd378153670 2023-03-03T06:26:36.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.242+0000 7fd374ff9700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd364064ee0 0x7fd364067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.242+0000 7fd374ff9700 1 -- 172.21.15.3:0/1012005992 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd36c083230 con 0x7fd378153670 2023-03-03T06:26:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.242+0000 7fd377fff700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd364064ee0 0x7fd364067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:36.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.246+0000 7fd377fff700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd364064ee0 0x7fd364067390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd368000f60 tx=0x7fd368007cc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.246+0000 7fd374ff9700 1 -- 172.21.15.3:0/1012005992 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd36c050fb0 con 0x7fd378153670 2023-03-03T06:26:36.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.458+0000 7fd37d09f700 1 -- 172.21.15.3:0/1012005992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3780b2910 con 0x7fd378153670 2023-03-03T06:26:36.466 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.458+0000 7fd374ff9700 1 -- 172.21.15.3:0/1012005992 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4635 (secure 0 0 0) 0x7fd36c0545d0 con 0x7fd378153670 2023-03-03T06:26:36.466 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:36.467 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/2744699972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2744699972},{"type":"v1","addr":"172.21.15.3:6834","nonce":2744699972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/708054272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":708054272},{"type":"v1","addr":"172.21.15.3:6835","nonce":708054272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2935923413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2935923413},{"type":"v1","addr":"172.21.15.32:6837","nonce":2935923413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-03T06:25:13.798672+0000","modified":"2023-03-03T06:25:41.755082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54179":{"gid":54179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/691220172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":691220172},{"type":"v1","addr":"172.21.15.32:6836","nonce":691220172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-03-03T06:26:36.470 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.466+0000 7fd37d09f700 1 -- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd364064ee0 msgr2=0x7fd364067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:36.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.466+0000 7fd37d09f700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd364064ee0 0x7fd364067390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd368000f60 tx=0x7fd368007cc0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.466+0000 7fd37d09f700 1 -- 172.21.15.3:0/1012005992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd378153670 msgr2=0x7fd3780ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:36.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.466+0000 7fd37d09f700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd378153670 0x7fd3780ca6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd36c014040 tx=0x7fd36c004770 comp rx=0 tx=0).stop 2023-03-03T06:26:36.471 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.466+0000 7fd37d09f700 1 -- 172.21.15.3:0/1012005992 shutdown_connections 2023-03-03T06:26:36.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.466+0000 7fd37d09f700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd364064ee0 0x7fd364067390 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.466+0000 7fd37d09f700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3781608d0 0x7fd3780b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.466+0000 7fd37d09f700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd378153670 0x7fd3780ca6f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.472 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.466+0000 7fd37d09f700 1 --2- 172.21.15.3:0/1012005992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3780cfb60 0x7fd37814cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.466+0000 7fd37d09f700 1 -- 172.21.15.3:0/1012005992 >> 172.21.15.3:0/1012005992 conn(0x7fd37806e5c0 msgr2=0x7fd37806eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:36.473 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.466+0000 7fd37d09f700 1 -- 172.21.15.3:0/1012005992 shutdown_connections 2023-03-03T06:26:36.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.466+0000 7fd37d09f700 1 -- 172.21.15.3:0/1012005992 wait complete. 2023-03-03T06:26:36.476 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2023-03-03T06:26:36.508 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:36.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.710+0000 7fcc85128700 1 Processor -- start 2023-03-03T06:26:36.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc85128700 1 -- start start 2023-03-03T06:26:36.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc85128700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcc800adf20 0x7fcc800b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc85128700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc8015dec0 0x7fcc8015e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc85128700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc800cb180 0x7fcc800cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc85128700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc800ae2f0 con 0x7fcc800adf20 2023-03-03T06:26:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc85128700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc800cbb30 con 0x7fcc8015dec0 2023-03-03T06:26:36.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc85128700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc800c9a20 con 0x7fcc800cb180 2023-03-03T06:26:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc7ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcc800adf20 0x7fcc800b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc777fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc8015dec0 0x7fcc8015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc84927700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc800cb180 0x7fcc800cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc7ffff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcc800adf20 0x7fcc800b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42082/0 (socket says 172.21.15.3:42082) 2023-03-03T06:26:36.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc777fe700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc8015dec0 0x7fcc8015e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:40164/0 (socket says 172.21.15.3:40164) 2023-03-03T06:26:36.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc7ffff700 1 -- 172.21.15.3:0/460983330 learned_addr learned my addr 172.21.15.3:0/460983330 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:36.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc777fe700 1 -- 172.21.15.3:0/460983330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc800cb180 msgr2=0x7fcc800cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:36.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.714+0000 7fcc777fe700 1 --2- 172.21.15.3:0/460983330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc800cb180 0x7fcc800cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc777fe700 1 -- 172.21.15.3:0/460983330 >> v2:172.21.15.3:3300/0 conn(0x7fcc800adf20 msgr2=0x7fcc800b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:36.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc777fe700 1 --2- 172.21.15.3:0/460983330 >> v2:172.21.15.3:3300/0 conn(0x7fcc800adf20 0x7fcc800b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc777fe700 1 -- 172.21.15.3:0/460983330 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc800c9fe0 con 0x7fcc8015dec0 2023-03-03T06:26:36.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc777fe700 1 --2- 172.21.15.3:0/460983330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc8015dec0 0x7fcc8015e310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcc800c93d0 tx=0x7fcc6800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5151a7b64aa4542 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:36.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc7f7fe700 1 -- 172.21.15.3:0/460983330 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6800e070 con 0x7fcc8015dec0 2023-03-03T06:26:36.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc7f7fe700 1 -- 172.21.15.3:0/460983330 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6800bb20 con 0x7fcc8015dec0 2023-03-03T06:26:36.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc7f7fe700 1 -- 172.21.15.3:0/460983330 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68012430 con 0x7fcc8015dec0 2023-03-03T06:26:36.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc85128700 1 -- 172.21.15.3:0/460983330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc8015dec0 msgr2=0x7fcc8015e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:36.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc85128700 1 --2- 172.21.15.3:0/460983330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc8015dec0 0x7fcc8015e310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcc800c93d0 tx=0x7fcc6800b330 comp rx=0 tx=0).stop 2023-03-03T06:26:36.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc85128700 1 -- 172.21.15.3:0/460983330 shutdown_connections 2023-03-03T06:26:36.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc85128700 1 --2- 172.21.15.3:0/460983330 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc800cb180 0x7fcc800cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc85128700 1 --2- 172.21.15.3:0/460983330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc8015dec0 0x7fcc8015e310 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc85128700 1 --2- 172.21.15.3:0/460983330 >> v2:172.21.15.3:3300/0 conn(0x7fcc800adf20 0x7fcc800b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc85128700 1 -- 172.21.15.3:0/460983330 >> 172.21.15.3:0/460983330 conn(0x7fcc80070a30 msgr2=0x7fcc80070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:36.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc85128700 1 -- 172.21.15.3:0/460983330 shutdown_connections 2023-03-03T06:26:36.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.718+0000 7fcc85128700 1 -- 172.21.15.3:0/460983330 wait complete. 2023-03-03T06:26:36.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.722+0000 7fcc85128700 1 Processor -- start 2023-03-03T06:26:36.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.722+0000 7fcc85128700 1 -- start start 2023-03-03T06:26:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc85128700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc800adf20 0x7fcc801557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc85128700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc800cb180 0x7fcc80157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc85128700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc8015dec0 0x7fcc80158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:36.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc85128700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc800b2c30 con 0x7fcc8015dec0 2023-03-03T06:26:36.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc85128700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80158650 con 0x7fcc800adf20 2023-03-03T06:26:36.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc85128700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80156640 con 0x7fcc800cb180 2023-03-03T06:26:36.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc84927700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc8015dec0 0x7fcc80158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc84927700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc8015dec0 0x7fcc80158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42086/0 (socket says 172.21.15.3:42086) 2023-03-03T06:26:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc84927700 1 -- 172.21.15.3:0/2271087219 learned_addr learned my addr 172.21.15.3:0/2271087219 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc7ffff700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc800adf20 0x7fcc801557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc777fe700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc800cb180 0x7fcc80157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc84927700 1 -- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc800cb180 msgr2=0x7fcc80157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc84927700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc800cb180 0x7fcc80157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc84927700 1 -- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc800adf20 msgr2=0x7fcc801557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc84927700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc800adf20 0x7fcc801557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc84927700 1 -- 172.21.15.3:0/2271087219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68009a70 con 0x7fcc8015dec0 2023-03-03T06:26:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc777fe700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc800cb180 0x7fcc80157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:36.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc84927700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc8015dec0 0x7fcc80158110 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800eb90 tx=0x7fcc7800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc7d7fa700 1 -- 172.21.15.3:0/2271087219 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78009740 con 0x7fcc8015dec0 2023-03-03T06:26:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc85128700 1 -- 172.21.15.3:0/2271087219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8015e910 con 0x7fcc8015dec0 2023-03-03T06:26:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc7d7fa700 1 -- 172.21.15.3:0/2271087219 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7800c850 con 0x7fcc8015dec0 2023-03-03T06:26:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc7d7fa700 1 -- 172.21.15.3:0/2271087219 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7800cbe0 con 0x7fcc8015dec0 2023-03-03T06:26:36.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.726+0000 7fcc85128700 1 -- 172.21.15.3:0/2271087219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8015a7d0 con 0x7fcc8015dec0 2023-03-03T06:26:36.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.730+0000 7fcc85128700 1 -- 172.21.15.3:0/2271087219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8015ebf0 con 0x7fcc8015dec0 2023-03-03T06:26:36.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.734+0000 7fcc7d7fa700 1 -- 172.21.15.3:0/2271087219 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fcc78019030 con 0x7fcc8015dec0 2023-03-03T06:26:36.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.734+0000 7fcc7d7fa700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fcc60064ee0 0x7fcc60067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:36.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.738+0000 7fcc7d7fa700 1 -- 172.21.15.3:0/2271087219 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcc780822d0 con 0x7fcc8015dec0 2023-03-03T06:26:36.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.738+0000 7fcc7ffff700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fcc60064ee0 0x7fcc60067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:36.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.738+0000 7fcc7ffff700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fcc60064ee0 0x7fcc60067390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009b80 tx=0x7fcc700092e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:36.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.742+0000 7fcc7d7fa700 1 -- 172.21.15.3:0/2271087219 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fcc7804ff70 con 0x7fcc8015dec0 2023-03-03T06:26:36.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.914+0000 7fcc85128700 1 -- 172.21.15.3:0/2271087219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc800b2de0 con 0x7fcc8015dec0 2023-03-03T06:26:36.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.918+0000 7fcc7d7fa700 1 -- 172.21.15.3:0/2271087219 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19652 (secure 0 0 0) 0x7fcc800b2de0 con 0x7fcc8015dec0 2023-03-03T06:26:36.925 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:36.925 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":30,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:25:53.877945+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:25:11.468534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-03T06:25:12.813283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-03T06:25:16.382800+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.10","mappings":[{"from":1,"to":5}]},{"pgid":"4.2f","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:36.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.922+0000 7fcc85128700 1 -- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fcc60064ee0 msgr2=0x7fcc60067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:36.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.922+0000 7fcc85128700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fcc60064ee0 0x7fcc60067390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009b80 tx=0x7fcc700092e0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.922+0000 7fcc85128700 1 -- 172.21.15.3:0/2271087219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc8015dec0 msgr2=0x7fcc80158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:36.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.922+0000 7fcc85128700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc8015dec0 0x7fcc80158110 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800eb90 tx=0x7fcc7800ef50 comp rx=0 tx=0).stop 2023-03-03T06:26:36.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.922+0000 7fcc85128700 1 -- 172.21.15.3:0/2271087219 shutdown_connections 2023-03-03T06:26:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.922+0000 7fcc85128700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fcc60064ee0 0x7fcc60067390 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.922+0000 7fcc85128700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc8015dec0 0x7fcc80158110 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.922+0000 7fcc85128700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcc800cb180 0x7fcc80157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.922+0000 7fcc85128700 1 --2- 172.21.15.3:0/2271087219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc800adf20 0x7fcc801557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.922+0000 7fcc85128700 1 -- 172.21.15.3:0/2271087219 >> 172.21.15.3:0/2271087219 conn(0x7fcc80070a30 msgr2=0x7fcc800ca470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.922+0000 7fcc85128700 1 -- 172.21.15.3:0/2271087219 shutdown_connections 2023-03-03T06:26:36.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:36.922+0000 7fcc85128700 1 -- 172.21.15.3:0/2271087219 wait complete. 2023-03-03T06:26:36.962 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-03-03T06:26:37.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.166+0000 7f87b14e8700 1 Processor -- start 2023-03-03T06:26:37.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.166+0000 7f87b14e8700 1 -- start start 2023-03-03T06:26:37.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87b14e8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87ac0aee70 0x7f87ac158440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:37.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87b14e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87ac15c040 0x7f87ac15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:37.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87b14e8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ac0cdd30 con 0x7f87ac06a780 2023-03-03T06:26:37.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87b14e8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ac15ca20 con 0x7f87ac0aee70 2023-03-03T06:26:37.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87b14e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac158980 con 0x7f87ac15c040 2023-03-03T06:26:37.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87a37fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87ac0aee70 0x7f87ac158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:37.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87abfff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f87ac06a780 0x7f87ac0c9a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:58324/0 (socket says 172.21.15.3:58324) 2023-03-03T06:26:37.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87abfff700 1 -- 172.21.15.3:0/2436617075 learned_addr learned my addr 172.21.15.3:0/2436617075 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:37.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87b0ce7700 1 --2- 172.21.15.3:0/2436617075 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87ac15c040 0x7f87ac15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:37.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87ab7fe700 1 -- 172.21.15.3:0/2436617075 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830417188 0 0) 0x7f87ac0cdd30 con 0x7f87ac06a780 2023-03-03T06:26:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87ab7fe700 1 -- 172.21.15.3:0/2436617075 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87980035d0 con 0x7f87ac06a780 2023-03-03T06:26:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87a37fe700 1 -- 172.21.15.3:0/2436617075 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87ac15c040 msgr2=0x7f87ac15c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87a37fe700 1 --2- 172.21.15.3:0/2436617075 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87ac15c040 0x7f87ac15c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.170+0000 7f87a37fe700 1 -- 172.21.15.3:0/2436617075 >> v1:172.21.15.32:6789/0 conn(0x7f87ac06a780 legacy=0x7f87ac0c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.174+0000 7f87a37fe700 1 -- 172.21.15.3:0/2436617075 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac0ca4d0 con 0x7f87ac0aee70 2023-03-03T06:26:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.174+0000 7f87a37fe700 1 --2- 172.21.15.3:0/2436617075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac0aee70 0x7f87ac158440 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8794009850 tx=0x7f8794009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b15fa781388394f server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.174+0000 7f87ab7fe700 1 -- 172.21.15.3:0/2436617075 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794011830 con 0x7f87ac0aee70 2023-03-03T06:26:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.174+0000 7f87ab7fe700 1 -- 172.21.15.3:0/2436617075 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8794011990 con 0x7f87ac0aee70 2023-03-03T06:26:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.174+0000 7f87ab7fe700 1 -- 172.21.15.3:0/2436617075 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794011d40 con 0x7f87ac0aee70 2023-03-03T06:26:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.174+0000 7f87b14e8700 1 -- 172.21.15.3:0/2436617075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac0aee70 msgr2=0x7f87ac158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.174+0000 7f87b14e8700 1 --2- 172.21.15.3:0/2436617075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac0aee70 0x7f87ac158440 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8794009850 tx=0x7f8794009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.174+0000 7f87b14e8700 1 -- 172.21.15.3:0/2436617075 shutdown_connections 2023-03-03T06:26:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.174+0000 7f87b14e8700 1 --2- 172.21.15.3:0/2436617075 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87ac15c040 0x7f87ac15c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.174+0000 7f87b14e8700 1 --2- 172.21.15.3:0/2436617075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac0aee70 0x7f87ac158440 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.174+0000 7f87b14e8700 1 -- 172.21.15.3:0/2436617075 >> 172.21.15.3:0/2436617075 conn(0x7f87ac070a30 msgr2=0x7f87ac070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.174+0000 7f87b14e8700 1 -- 172.21.15.3:0/2436617075 shutdown_connections 2023-03-03T06:26:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.174+0000 7f87b14e8700 1 -- 172.21.15.3:0/2436617075 wait complete. 2023-03-03T06:26:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.178+0000 7f87b14e8700 1 Processor -- start 2023-03-03T06:26:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.178+0000 7f87b14e8700 1 -- start start 2023-03-03T06:26:37.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b14e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87ac06a780 0x7f87ac149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:37.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b14e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87ac0aee70 0x7f87ac14c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:37.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b14e8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac15c040 0x7f87ac147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:37.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b14e8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac149230 con 0x7f87ac15c040 2023-03-03T06:26:37.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b14e8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac14c850 con 0x7f87ac0aee70 2023-03-03T06:26:37.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b14e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac14cb30 con 0x7f87ac06a780 2023-03-03T06:26:37.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b0ce7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac15c040 0x7f87ac147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:37.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b0ce7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac15c040 0x7f87ac147e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42112/0 (socket says 172.21.15.3:42112) 2023-03-03T06:26:37.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b0ce7700 1 -- 172.21.15.3:0/3146581569 learned_addr learned my addr 172.21.15.3:0/3146581569 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:37.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87a37fe700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87ac0aee70 0x7f87ac14c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:37.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87abfff700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87ac06a780 0x7f87ac149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:37.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b0ce7700 1 -- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87ac06a780 msgr2=0x7f87ac149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:37.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b0ce7700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87ac06a780 0x7f87ac149640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:37.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b0ce7700 1 -- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87ac0aee70 msgr2=0x7f87ac14c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:37.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b0ce7700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87ac0aee70 0x7f87ac14c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:37.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b0ce7700 1 -- 172.21.15.3:0/3146581569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87940094b0 con 0x7f87ac15c040 2023-03-03T06:26:37.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.182+0000 7f87b0ce7700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac15c040 0x7f87ac147e80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f87a400efc0 tx=0x7f87a400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:37.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.186+0000 7f87a97fa700 1 -- 172.21.15.3:0/3146581569 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a400cb80 con 0x7f87ac15c040 2023-03-03T06:26:37.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.186+0000 7f87a97fa700 1 -- 172.21.15.3:0/3146581569 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a400f040 con 0x7f87ac15c040 2023-03-03T06:26:37.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.186+0000 7f87a97fa700 1 -- 172.21.15.3:0/3146581569 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4014480 con 0x7f87ac15c040 2023-03-03T06:26:37.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.186+0000 7f87b14e8700 1 -- 172.21.15.3:0/3146581569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ac0cd9c0 con 0x7f87ac15c040 2023-03-03T06:26:37.191 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.186+0000 7f87b14e8700 1 -- 172.21.15.3:0/3146581569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ac0cd6e0 con 0x7f87ac15c040 2023-03-03T06:26:37.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.186+0000 7f87b14e8700 1 -- 172.21.15.3:0/3146581569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ac0ca7b0 con 0x7f87ac15c040 2023-03-03T06:26:37.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.190+0000 7f87a97fa700 1 -- 172.21.15.3:0/3146581569 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f87a40145e0 con 0x7f87ac15c040 2023-03-03T06:26:37.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.190+0000 7f87a97fa700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f878c064f10 0x7f878c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:37.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.190+0000 7f87a97fa700 1 -- 172.21.15.3:0/3146581569 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f87a4082c30 con 0x7f87ac15c040 2023-03-03T06:26:37.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.194+0000 7f87abfff700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f878c064f10 0x7f878c0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:37.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.194+0000 7f87abfff700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f878c064f10 0x7f878c0673c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f879c0011e0 tx=0x7f879c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:37.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.198+0000 7f87a97fa700 1 -- 172.21.15.3:0/3146581569 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f87a4050930 con 0x7f87ac15c040 2023-03-03T06:26:37.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.406+0000 7f87b14e8700 1 -- 172.21.15.3:0/3146581569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f87ac0ccf10 con 0x7f87ac15c040 2023-03-03T06:26:37.850 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T06:26:37.846+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-03-03T06:26:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.854+0000 7f87a97fa700 1 -- 172.21.15.3:0/3146581569 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f87a4053f50 con 0x7f87ac15c040 2023-03-03T06:26:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.862+0000 7f87b14e8700 1 -- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f878c064f10 msgr2=0x7f878c0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.862+0000 7f87b14e8700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f878c064f10 0x7f878c0673c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f879c0011e0 tx=0x7f879c009040 comp rx=0 tx=0).stop 2023-03-03T06:26:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.862+0000 7f87b14e8700 1 -- 172.21.15.3:0/3146581569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac15c040 msgr2=0x7f87ac147e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:37.866 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.862+0000 7f87b14e8700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac15c040 0x7f87ac147e80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f87a400efc0 tx=0x7f87a400c0c0 comp rx=0 tx=0).stop 2023-03-03T06:26:37.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.862+0000 7f87b14e8700 1 -- 172.21.15.3:0/3146581569 shutdown_connections 2023-03-03T06:26:37.867 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.862+0000 7f87b14e8700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f878c064f10 0x7f878c0673c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.862+0000 7f87b14e8700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87ac15c040 0x7f87ac147e80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.862+0000 7f87b14e8700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87ac0aee70 0x7f87ac14c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.862+0000 7f87b14e8700 1 --2- 172.21.15.3:0/3146581569 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f87ac06a780 0x7f87ac149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.862+0000 7f87b14e8700 1 -- 172.21.15.3:0/3146581569 >> 172.21.15.3:0/3146581569 conn(0x7f87ac070a30 msgr2=0x7f87ac06bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.862+0000 7f87b14e8700 1 -- 172.21.15.3:0/3146581569 shutdown_connections 2023-03-03T06:26:37.869 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:37.862+0000 7f87b14e8700 1 -- 172.21.15.3:0/3146581569 wait complete. 2023-03-03T06:26:37.871 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-03-03T06:26:37.880 INFO:tasks.ceph.mds.b.smithi032.stderr: -5496> 2023-03-03T06:25:18.426+0000 7f60b0589700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-03-03T06:26:37.903 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-03-03T06:26:37.962 INFO:tasks.ceph.mds.b.smithi032.stdout:starting mds.b at 2023-03-03T06:26:38.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.118+0000 7ff0a248c700 1 Processor -- start 2023-03-03T06:26:38.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.118+0000 7ff0a248c700 1 -- start start 2023-03-03T06:26:38.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a248c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09c160790 0x7ff09c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:38.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a248c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff09c160b60 0x7ff09c0b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:38.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a248c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09c0ccc20 con 0x7ff09c0cbea0 2023-03-03T06:26:38.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a248c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c0cc380 con 0x7ff09c160b60 2023-03-03T06:26:38.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a248c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0cc660 con 0x7ff09c160790 2023-03-03T06:26:38.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a148a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09c160790 0x7ff09c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:38.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a0c89700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff09c160b60 0x7ff09c0b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:38.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a1c8b700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff09c0cbea0 0x7ff09c0709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:58328/0 (socket says 172.21.15.3:58328) 2023-03-03T06:26:38.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a1c8b700 1 -- 172.21.15.3:0/3291116635 learned_addr learned my addr 172.21.15.3:0/3291116635 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:38.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a0c89700 1 -- 172.21.15.3:0/3291116635 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09c160790 msgr2=0x7ff09c0ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:38.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a0c89700 1 --2- 172.21.15.3:0/3291116635 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09c160790 0x7ff09c0ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:38.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a0c89700 1 -- 172.21.15.3:0/3291116635 >> v1:172.21.15.32:6789/0 conn(0x7ff09c0cbea0 legacy=0x7ff09c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:38.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a0c89700 1 -- 172.21.15.3:0/3291116635 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0cc940 con 0x7ff09c160b60 2023-03-03T06:26:38.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.122+0000 7ff0a148a700 1 --2- 172.21.15.3:0/3291116635 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09c160790 0x7ff09c0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:38.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.126+0000 7ff0a0c89700 1 --2- 172.21.15.3:0/3291116635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09c160b60 0x7ff09c0b07c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009850 tx=0x7ff08c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd9020ca0c74f044 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:38.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.126+0000 7ff093fff700 1 -- 172.21.15.3:0/3291116635 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c011830 con 0x7ff09c160b60 2023-03-03T06:26:38.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.126+0000 7ff093fff700 1 -- 172.21.15.3:0/3291116635 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c011990 con 0x7ff09c160b60 2023-03-03T06:26:38.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.126+0000 7ff093fff700 1 -- 172.21.15.3:0/3291116635 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c011d40 con 0x7ff09c160b60 2023-03-03T06:26:38.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.126+0000 7ff0a248c700 1 -- 172.21.15.3:0/3291116635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09c160b60 msgr2=0x7ff09c0b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:38.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.126+0000 7ff0a248c700 1 --2- 172.21.15.3:0/3291116635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09c160b60 0x7ff09c0b07c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009850 tx=0x7ff08c009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:38.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.126+0000 7ff0a248c700 1 -- 172.21.15.3:0/3291116635 shutdown_connections 2023-03-03T06:26:38.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.126+0000 7ff0a248c700 1 --2- 172.21.15.3:0/3291116635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09c160b60 0x7ff09c0b07c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:38.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.126+0000 7ff0a248c700 1 --2- 172.21.15.3:0/3291116635 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09c160790 0x7ff09c0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:38.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.126+0000 7ff0a248c700 1 -- 172.21.15.3:0/3291116635 >> 172.21.15.3:0/3291116635 conn(0x7ff09c0607c0 msgr2=0x7ff09c060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:38.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.130+0000 7ff0a248c700 1 -- 172.21.15.3:0/3291116635 shutdown_connections 2023-03-03T06:26:38.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.130+0000 7ff0a248c700 1 -- 172.21.15.3:0/3291116635 wait complete. 2023-03-03T06:26:38.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.130+0000 7ff0a248c700 1 Processor -- start 2023-03-03T06:26:38.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.130+0000 7ff0a248c700 1 -- start start 2023-03-03T06:26:38.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a248c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09c0cbea0 0x7ff09c155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:38.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a248c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09c160790 0x7ff09c0c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:38.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a248c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09c160b60 0x7ff09c161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a248c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c163e30 con 0x7ff09c160790 2023-03-03T06:26:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a248c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0c8710 con 0x7ff09c0cbea0 2023-03-03T06:26:38.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a248c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c153670 con 0x7ff09c160b60 2023-03-03T06:26:38.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a0c89700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09c160790 0x7ff09c0c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:38.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a0c89700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09c160790 0x7ff09c0c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42122/0 (socket says 172.21.15.3:42122) 2023-03-03T06:26:38.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a0c89700 1 -- 172.21.15.3:0/78444867 learned_addr learned my addr 172.21.15.3:0/78444867 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:38.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a148a700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09c0cbea0 0x7ff09c155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:38.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a1c8b700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09c160b60 0x7ff09c161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:38.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a0c89700 1 -- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09c160b60 msgr2=0x7ff09c161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a0c89700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09c160b60 0x7ff09c161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a0c89700 1 -- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09c0cbea0 msgr2=0x7ff09c155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a0c89700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09c0cbea0 0x7ff09c155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a0c89700 1 -- 172.21.15.3:0/78444867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0980094b0 con 0x7ff09c160790 2023-03-03T06:26:38.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a148a700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09c0cbea0 0x7ff09c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:38.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a0c89700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09c160790 0x7ff09c0c81d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0090b0 tx=0x7ff08c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:38.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff091ffb700 1 -- 172.21.15.3:0/78444867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0036c0 con 0x7ff09c160790 2023-03-03T06:26:38.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff091ffb700 1 -- 172.21.15.3:0/78444867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c01e070 con 0x7ff09c160790 2023-03-03T06:26:38.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff091ffb700 1 -- 172.21.15.3:0/78444867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c003a80 con 0x7ff09c160790 2023-03-03T06:26:38.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a248c700 1 -- 172.21.15.3:0/78444867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08c0094b0 con 0x7ff09c160790 2023-03-03T06:26:38.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.134+0000 7ff0a248c700 1 -- 172.21.15.3:0/78444867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c0cb980 con 0x7ff09c160790 2023-03-03T06:26:38.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.138+0000 7ff0a248c700 1 -- 172.21.15.3:0/78444867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c0cb6a0 con 0x7ff09c160790 2023-03-03T06:26:38.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.138+0000 7ff091ffb700 1 -- 172.21.15.3:0/78444867 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff08c00f050 con 0x7ff09c160790 2023-03-03T06:26:38.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.142+0000 7ff091ffb700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff080064ee0 0x7ff080067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:38.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.142+0000 7ff091ffb700 1 -- 172.21.15.3:0/78444867 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6957+0+0 (secure 0 0 0) 0x7ff08c020070 con 0x7ff09c160790 2023-03-03T06:26:38.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.142+0000 7ff0a148a700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff080064ee0 0x7ff080067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:38.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.146+0000 7ff0a148a700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff080064ee0 0x7ff080067390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff098009dd0 tx=0x7ff098008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:38.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.146+0000 7ff091ffb700 1 -- 172.21.15.3:0/78444867 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff08c051610 con 0x7ff09c160790 2023-03-03T06:26:38.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.366+0000 7ff0a248c700 1 -- 172.21.15.3:0/78444867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff09c0c1580 con 0x7ff09c160790 2023-03-03T06:26:38.880 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.874+0000 7ff091ffb700 1 -- 172.21.15.3:0/78444867 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff08c054c30 con 0x7ff09c160790 2023-03-03T06:26:38.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.882+0000 7ff0a248c700 1 -- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff080064ee0 msgr2=0x7ff080067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.882+0000 7ff0a248c700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff080064ee0 0x7ff080067390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff098009dd0 tx=0x7ff098008040 comp rx=0 tx=0).stop 2023-03-03T06:26:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.882+0000 7ff0a248c700 1 -- 172.21.15.3:0/78444867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09c160790 msgr2=0x7ff09c0c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.882+0000 7ff0a248c700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09c160790 0x7ff09c0c81d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0090b0 tx=0x7ff08c004770 comp rx=0 tx=0).stop 2023-03-03T06:26:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.882+0000 7ff0a248c700 1 -- 172.21.15.3:0/78444867 shutdown_connections 2023-03-03T06:26:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.882+0000 7ff0a248c700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff080064ee0 0x7ff080067390 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.882+0000 7ff0a248c700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff09c160b60 0x7ff09c161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.882+0000 7ff0a248c700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff09c160790 0x7ff09c0c81d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.882+0000 7ff0a248c700 1 --2- 172.21.15.3:0/78444867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff09c0cbea0 0x7ff09c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.882+0000 7ff0a248c700 1 -- 172.21.15.3:0/78444867 >> 172.21.15.3:0/78444867 conn(0x7ff09c0607c0 msgr2=0x7ff09c0c93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.882+0000 7ff0a248c700 1 -- 172.21.15.3:0/78444867 shutdown_connections 2023-03-03T06:26:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:38.882+0000 7ff0a248c700 1 -- 172.21.15.3:0/78444867 wait complete. 2023-03-03T06:26:38.924 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-03-03T06:26:39.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.134+0000 7f0a09098700 1 Processor -- start 2023-03-03T06:26:39.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.134+0000 7f0a09098700 1 -- start start 2023-03-03T06:26:39.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.138+0000 7f0a09098700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0a040b4d90 0x7f0a040b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:39.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.138+0000 7f0a09098700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a040cd2f0 con 0x7f0a0415faf0 2023-03-03T06:26:39.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.138+0000 7f0a09098700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a040cf240 con 0x7f0a040cd7d0 2023-03-03T06:26:39.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.138+0000 7f0a09098700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a041609d0 con 0x7f0a040b4d90 2023-03-03T06:26:39.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.138+0000 7f0a03fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0a0415faf0 0x7f0a040b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43148/0 (socket says 172.21.15.3:43148) 2023-03-03T06:26:39.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.138+0000 7f0a03fff700 1 -- 172.21.15.3:0/2697621700 learned_addr learned my addr 172.21.15.3:0/2697621700 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:39.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f09fb7fe700 1 --2- 172.21.15.3:0/2697621700 >> v2:172.21.15.3:3300/0 conn(0x7f0a040b4d90 0x7f0a040b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:39.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287005337 0 0) 0x7f0a040cd2f0 con 0x7f0a0415faf0 2023-03-03T06:26:39.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f00035d0 con 0x7f0a0415faf0 2023-03-03T06:26:39.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336405381 0 0) 0x7f0a040cf240 con 0x7f0a040cd7d0 2023-03-03T06:26:39.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a040cd2f0 con 0x7f0a040cd7d0 2023-03-03T06:26:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1099033887 0 0) 0x7f09f00035d0 con 0x7f0a0415faf0 2023-03-03T06:26:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 >> v2:172.21.15.3:3300/0 conn(0x7f0a040b4d90 msgr2=0x7f0a040b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 --2- 172.21.15.3:0/2697621700 >> v2:172.21.15.3:3300/0 conn(0x7f0a040b4d90 0x7f0a040b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 >> v1:172.21.15.32:6789/0 conn(0x7f0a040cd7d0 legacy=0x7f0a040d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a04160bc0 con 0x7f0a0415faf0 2023-03-03T06:26:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f09f4001670 con 0x7f0a0415faf0 2023-03-03T06:26:39.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 >> v1:172.21.15.3:6789/0 conn(0x7f0a0415faf0 legacy=0x7f0a040b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 --2- 172.21.15.3:0/2697621700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f0003bb0 0x7f09f0003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 --2- 172.21.15.3:0/2697621700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f0004870 0x7f09f0004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 --2- 172.21.15.3:0/2697621700 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09f0005220 0x7f09f000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f000c050 con 0x7f09f0004870 2023-03-03T06:26:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f000c1c0 con 0x7f09f0003bb0 2023-03-03T06:26:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f000c380 con 0x7f09f0005220 2023-03-03T06:26:39.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a03fff700 1 --2- 172.21.15.3:0/2697621700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f0003bb0 0x7f09f0003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:39.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f09fb7fe700 1 --2- 172.21.15.3:0/2697621700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f0004870 0x7f09f0004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:39.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a08897700 1 --2- 172.21.15.3:0/2697621700 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09f0005220 0x7f09f000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:39.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a03fff700 1 -- 172.21.15.3:0/2697621700 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09f0005220 msgr2=0x7f09f000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:39.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a03fff700 1 --2- 172.21.15.3:0/2697621700 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09f0005220 0x7f09f000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:39.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a03fff700 1 -- 172.21.15.3:0/2697621700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f0004870 msgr2=0x7f09f0004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:39.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a03fff700 1 --2- 172.21.15.3:0/2697621700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09f0004870 0x7f09f0004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:39.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a03fff700 1 -- 172.21.15.3:0/2697621700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f00035d0 con 0x7f09f0003bb0 2023-03-03T06:26:39.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.142+0000 7f0a08897700 1 -- 172.21.15.3:0/2697621700 reap_dead start 2023-03-03T06:26:39.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.146+0000 7f0a03fff700 1 --2- 172.21.15.3:0/2697621700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f0003bb0 0x7f09f0003fa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f09f400c210 tx=0x7f09f00058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc851efedd5ae566 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:39.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.146+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f400cdb0 con 0x7f09f0003bb0 2023-03-03T06:26:39.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.146+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f4005980 con 0x7f09f0003bb0 2023-03-03T06:26:39.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.146+0000 7f0a037fe700 1 -- 172.21.15.3:0/2697621700 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4005d30 con 0x7f09f0003bb0 2023-03-03T06:26:39.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.146+0000 7f0a09098700 1 -- 172.21.15.3:0/2697621700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f0003bb0 msgr2=0x7f09f0003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:39.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.146+0000 7f0a09098700 1 --2- 172.21.15.3:0/2697621700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f0003bb0 0x7f09f0003fa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f09f400c210 tx=0x7f09f00058a0 comp rx=0 tx=0).stop 2023-03-03T06:26:39.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.146+0000 7f0a09098700 1 -- 172.21.15.3:0/2697621700 shutdown_connections 2023-03-03T06:26:39.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.146+0000 7f0a09098700 1 --2- 172.21.15.3:0/2697621700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f0003bb0 0x7f09f0003fa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:39.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.146+0000 7f0a09098700 1 -- 172.21.15.3:0/2697621700 >> 172.21.15.3:0/2697621700 conn(0x7f0a040cf680 msgr2=0x7f0a040cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:39.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.146+0000 7f0a09098700 1 -- 172.21.15.3:0/2697621700 shutdown_connections 2023-03-03T06:26:39.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.146+0000 7f0a09098700 1 -- 172.21.15.3:0/2697621700 wait complete. 2023-03-03T06:26:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.150+0000 7f0a09098700 1 Processor -- start 2023-03-03T06:26:39.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.150+0000 7f0a09098700 1 -- start start 2023-03-03T06:26:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.150+0000 7f0a09098700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09f0003bb0 0x7f0a040c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:39.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.150+0000 7f0a09098700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a04153e80 0x7f0a040c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:39.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.150+0000 7f0a09098700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a040cfa50 0x7f0a040cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:39.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.150+0000 7f0a09098700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040c8f40 con 0x7f0a04153e80 2023-03-03T06:26:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.150+0000 7f0a09098700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040d0400 con 0x7f0a040cfa50 2023-03-03T06:26:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.150+0000 7f0a09098700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040c7ff0 con 0x7f09f0003bb0 2023-03-03T06:26:39.161 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f09fb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a04153e80 0x7f0a040c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f09fb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a04153e80 0x7f0a040c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42148/0 (socket says 172.21.15.3:42148) 2023-03-03T06:26:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f09fb7fe700 1 -- 172.21.15.3:0/1504550612 learned_addr learned my addr 172.21.15.3:0/1504550612 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f0a03fff700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09f0003bb0 0x7f0a040c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:39.162 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f0a08897700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a040cfa50 0x7f0a040cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f09fb7fe700 1 -- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09f0003bb0 msgr2=0x7f0a040c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f09fb7fe700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09f0003bb0 0x7f0a040c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:39.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f09fb7fe700 1 -- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a040cfa50 msgr2=0x7f0a040cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:39.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f09fb7fe700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a040cfa50 0x7f0a040cfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:39.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f09fb7fe700 1 -- 172.21.15.3:0/1504550612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040ceef0 con 0x7f0a04153e80 2023-03-03T06:26:39.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f0a08897700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a040cfa50 0x7f0a040cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:39.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f09fb7fe700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a04153e80 0x7f0a040c7ab0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f09ec011900 tx=0x7f09ec011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:39.164 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f0a017fa700 1 -- 172.21.15.3:0/1504550612 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec011e50 con 0x7f0a04153e80 2023-03-03T06:26:39.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f0a017fa700 1 -- 172.21.15.3:0/1504550612 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec004240 con 0x7f0a04153e80 2023-03-03T06:26:39.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f0a017fa700 1 -- 172.21.15.3:0/1504550612 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0045b0 con 0x7f0a04153e80 2023-03-03T06:26:39.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f0a09098700 1 -- 172.21.15.3:0/1504550612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a04160ea0 con 0x7f0a04153e80 2023-03-03T06:26:39.165 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.154+0000 7f0a09098700 1 -- 172.21.15.3:0/1504550612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a040b1ac0 con 0x7f0a04153e80 2023-03-03T06:26:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.158+0000 7f0a09098700 1 -- 172.21.15.3:0/1504550612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a040677c0 con 0x7f0a04153e80 2023-03-03T06:26:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.158+0000 7f0a017fa700 1 -- 172.21.15.3:0/1504550612 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f09ec025070 con 0x7f0a04153e80 2023-03-03T06:26:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.158+0000 7f0a017fa700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f09e4064ee0 0x7f09e4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.158+0000 7f0a017fa700 1 -- 172.21.15.3:0/1504550612 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f09ec0826a0 con 0x7f0a04153e80 2023-03-03T06:26:39.166 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.162+0000 7f0a03fff700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f09e4064ee0 0x7f09e4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:39.168 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.162+0000 7f0a03fff700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f09e4064ee0 0x7f09e4067390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f09f40018b0 tx=0x7f09f400a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:39.174 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.170+0000 7f0a017fa700 1 -- 172.21.15.3:0/1504550612 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f09ec050340 con 0x7f0a04153e80 2023-03-03T06:26:39.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.342+0000 7f0a09098700 1 -- 172.21.15.3:0/1504550612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0a04073e90 con 0x7f0a04153e80 2023-03-03T06:26:39.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.986+0000 7f0a017fa700 1 -- 172.21.15.3:0/1504550612 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v32) v1 ==== 181+0+0 (secure 0 0 0) 0x7f09ec0230d0 con 0x7f0a04153e80 2023-03-03T06:26:39.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.994+0000 7f0a09098700 1 -- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f09e4064ee0 msgr2=0x7f09e4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:40.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.994+0000 7f0a09098700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f09e4064ee0 0x7f09e4067390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f09f40018b0 tx=0x7f09f400a000 comp rx=0 tx=0).stop 2023-03-03T06:26:40.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.994+0000 7f0a09098700 1 -- 172.21.15.3:0/1504550612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a04153e80 msgr2=0x7f0a040c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:40.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.994+0000 7f0a09098700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a04153e80 0x7f0a040c7ab0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f09ec011900 tx=0x7f09ec011c10 comp rx=0 tx=0).stop 2023-03-03T06:26:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.994+0000 7f0a09098700 1 -- 172.21.15.3:0/1504550612 shutdown_connections 2023-03-03T06:26:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.994+0000 7f0a09098700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f09e4064ee0 0x7f09e4067390 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.994+0000 7f0a09098700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a040cfa50 0x7f0a040cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.994+0000 7f0a09098700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a04153e80 0x7f0a040c7ab0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.994+0000 7f0a09098700 1 --2- 172.21.15.3:0/1504550612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f09f0003bb0 0x7f0a040c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.994+0000 7f0a09098700 1 -- 172.21.15.3:0/1504550612 >> 172.21.15.3:0/1504550612 conn(0x7f0a040cf680 msgr2=0x7f0a040b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.994+0000 7f0a09098700 1 -- 172.21.15.3:0/1504550612 shutdown_connections 2023-03-03T06:26:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:39.994+0000 7f0a09098700 1 -- 172.21.15.3:0/1504550612 wait complete. 2023-03-03T06:26:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2023-03-03T06:26:40.035 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-03-03T06:26:40.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.250+0000 7fbe191b4700 1 Processor -- start 2023-03-03T06:26:40.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.250+0000 7fbe191b4700 1 -- start start 2023-03-03T06:26:40.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.254+0000 7fbe191b4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe140d0b30 0x7fbe140d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:40.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.254+0000 7fbe191b4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe140d14d0 con 0x7fbe1415d6e0 2023-03-03T06:26:40.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.254+0000 7fbe191b4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe140cf3b0 con 0x7fbe14156ba0 2023-03-03T06:26:40.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.254+0000 7fbe191b4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe14156f70 con 0x7fbe140d0b30 2023-03-03T06:26:40.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.254+0000 7fbe137fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbe1415d6e0 0x7fbe140cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43160/0 (socket says 172.21.15.3:43160) 2023-03-03T06:26:40.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.254+0000 7fbe137fe700 1 -- 172.21.15.3:0/3117780961 learned_addr learned my addr 172.21.15.3:0/3117780961 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:40.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.254+0000 7fbe189b3700 1 --2- 172.21.15.3:0/3117780961 >> v2:172.21.15.3:3300/0 conn(0x7fbe140d0b30 0x7fbe140d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:40.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.254+0000 7fbe12ffd700 1 -- 172.21.15.3:0/3117780961 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752919956 0 0) 0x7fbe140d14d0 con 0x7fbe1415d6e0 2023-03-03T06:26:40.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.254+0000 7fbe12ffd700 1 -- 172.21.15.3:0/3117780961 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdfc0035d0 con 0x7fbe1415d6e0 2023-03-03T06:26:40.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe189b3700 1 -- 172.21.15.3:0/3117780961 >> v1:172.21.15.32:6789/0 conn(0x7fbe14156ba0 legacy=0x7fbe14071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:40.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe189b3700 1 -- 172.21.15.3:0/3117780961 >> v1:172.21.15.3:6789/0 conn(0x7fbe1415d6e0 legacy=0x7fbe140cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:40.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe189b3700 1 -- 172.21.15.3:0/3117780961 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe140cd320 con 0x7fbe140d0b30 2023-03-03T06:26:40.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe189b3700 1 --2- 172.21.15.3:0/3117780961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe140d0b30 0x7fbe140d0f90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009d90 tx=0x7fbe0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62b08d42df91992 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:40.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe12ffd700 1 -- 172.21.15.3:0/3117780961 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c00e070 con 0x7fbe140d0b30 2023-03-03T06:26:40.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe12ffd700 1 -- 172.21.15.3:0/3117780961 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0c00bb20 con 0x7fbe140d0b30 2023-03-03T06:26:40.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe12ffd700 1 -- 172.21.15.3:0/3117780961 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c012430 con 0x7fbe140d0b30 2023-03-03T06:26:40.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe191b4700 1 -- 172.21.15.3:0/3117780961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe140d0b30 msgr2=0x7fbe140d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:40.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe191b4700 1 --2- 172.21.15.3:0/3117780961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe140d0b30 0x7fbe140d0f90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009d90 tx=0x7fbe0c00b330 comp rx=0 tx=0).stop 2023-03-03T06:26:40.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe191b4700 1 -- 172.21.15.3:0/3117780961 shutdown_connections 2023-03-03T06:26:40.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe191b4700 1 --2- 172.21.15.3:0/3117780961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe140d0b30 0x7fbe140d0f90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:40.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe191b4700 1 -- 172.21.15.3:0/3117780961 >> 172.21.15.3:0/3117780961 conn(0x7fbe1406e5c0 msgr2=0x7fbe1406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:40.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe191b4700 1 -- 172.21.15.3:0/3117780961 shutdown_connections 2023-03-03T06:26:40.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.258+0000 7fbe191b4700 1 -- 172.21.15.3:0/3117780961 wait complete. 2023-03-03T06:26:40.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.262+0000 7fbe191b4700 1 Processor -- start 2023-03-03T06:26:40.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.262+0000 7fbe191b4700 1 -- start start 2023-03-03T06:26:40.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.262+0000 7fbe191b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe140d0b30 0x7fbe140b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:40.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.262+0000 7fbe191b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe14156ba0 0x7fbe14154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:40.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.262+0000 7fbe191b4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe1415d6e0 0x7fbe140b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:40.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.262+0000 7fbe191b4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe14155270 con 0x7fbe140d0b30 2023-03-03T06:26:40.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.262+0000 7fbe191b4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe140b60d0 con 0x7fbe14156ba0 2023-03-03T06:26:40.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.262+0000 7fbe191b4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe140b63b0 con 0x7fbe1415d6e0 2023-03-03T06:26:40.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.262+0000 7fbe13fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe140d0b30 0x7fbe140b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:40.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.262+0000 7fbe13fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe140d0b30 0x7fbe140b5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57506/0 (socket says 172.21.15.3:57506) 2023-03-03T06:26:40.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.262+0000 7fbe13fff700 1 -- 172.21.15.3:0/2798448993 learned_addr learned my addr 172.21.15.3:0/2798448993 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:40.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe137fe700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe14156ba0 0x7fbe14154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:40.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe189b3700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe1415d6e0 0x7fbe140b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:40.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe13fff700 1 -- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe1415d6e0 msgr2=0x7fbe140b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:40.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe13fff700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe1415d6e0 0x7fbe140b7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:40.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe13fff700 1 -- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe14156ba0 msgr2=0x7fbe14154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:40.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe13fff700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe14156ba0 0x7fbe14154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:40.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe13fff700 1 -- 172.21.15.3:0/2798448993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c009a70 con 0x7fbe140d0b30 2023-03-03T06:26:40.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe13fff700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe140d0b30 0x7fbe140b5a60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400c500 tx=0x7fbe0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:40.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe10ff9700 1 -- 172.21.15.3:0/2798448993 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe040052d0 con 0x7fbe140d0b30 2023-03-03T06:26:40.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe10ff9700 1 -- 172.21.15.3:0/2798448993 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe04005430 con 0x7fbe140d0b30 2023-03-03T06:26:40.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe191b4700 1 -- 172.21.15.3:0/2798448993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe140cd8e0 con 0x7fbe140d0b30 2023-03-03T06:26:40.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe10ff9700 1 -- 172.21.15.3:0/2798448993 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe040055a0 con 0x7fbe140d0b30 2023-03-03T06:26:40.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe191b4700 1 -- 172.21.15.3:0/2798448993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe140cd600 con 0x7fbe140d0b30 2023-03-03T06:26:40.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.266+0000 7fbe191b4700 1 -- 172.21.15.3:0/2798448993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe140cff50 con 0x7fbe140d0b30 2023-03-03T06:26:40.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.270+0000 7fbe10ff9700 1 -- 172.21.15.3:0/2798448993 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fbe04019030 con 0x7fbe140d0b30 2023-03-03T06:26:40.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.274+0000 7fbe10ff9700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbe00064f10 0x7fbe000673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:40.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.274+0000 7fbe10ff9700 1 -- 172.21.15.3:0/2798448993 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fbe04081540 con 0x7fbe140d0b30 2023-03-03T06:26:40.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.274+0000 7fbe137fe700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbe00064f10 0x7fbe000673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:40.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.274+0000 7fbe137fe700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbe00064f10 0x7fbe000673c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbe14156180 tx=0x7fbe08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:40.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.278+0000 7fbe10ff9700 1 -- 172.21.15.3:0/2798448993 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fbe0404f3c0 con 0x7fbe140d0b30 2023-03-03T06:26:40.460 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.454+0000 7fbe191b4700 1 -- 172.21.15.3:0/2798448993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbe141557e0 con 0x7fbe140d0b30 2023-03-03T06:26:41.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:40.998+0000 7fbe10ff9700 1 -- 172.21.15.3:0/2798448993 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v33) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbe040529e0 con 0x7fbe140d0b30 2023-03-03T06:26:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.006+0000 7fbe191b4700 1 -- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbe00064f10 msgr2=0x7fbe000673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.006+0000 7fbe191b4700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbe00064f10 0x7fbe000673c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbe14156180 tx=0x7fbe08009040 comp rx=0 tx=0).stop 2023-03-03T06:26:41.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.006+0000 7fbe191b4700 1 -- 172.21.15.3:0/2798448993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe140d0b30 msgr2=0x7fbe140b5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.006+0000 7fbe191b4700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe140d0b30 0x7fbe140b5a60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400c500 tx=0x7fbe0400c8c0 comp rx=0 tx=0).stop 2023-03-03T06:26:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.006+0000 7fbe191b4700 1 -- 172.21.15.3:0/2798448993 shutdown_connections 2023-03-03T06:26:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.006+0000 7fbe191b4700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbe00064f10 0x7fbe000673c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.006+0000 7fbe191b4700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbe1415d6e0 0x7fbe140b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.006+0000 7fbe191b4700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe14156ba0 0x7fbe14154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.006+0000 7fbe191b4700 1 --2- 172.21.15.3:0/2798448993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe140d0b30 0x7fbe140b5a60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:41.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.006+0000 7fbe191b4700 1 -- 172.21.15.3:0/2798448993 >> 172.21.15.3:0/2798448993 conn(0x7fbe1406e5c0 msgr2=0x7fbe1415f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:41.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.006+0000 7fbe191b4700 1 -- 172.21.15.3:0/2798448993 shutdown_connections 2023-03-03T06:26:41.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.006+0000 7fbe191b4700 1 -- 172.21.15.3:0/2798448993 wait complete. 2023-03-03T06:26:41.016 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2023-03-03T06:26:41.047 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-03-03T06:26:41.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.258+0000 7f18e1928700 1 Processor -- start 2023-03-03T06:26:41.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.258+0000 7f18e1928700 1 -- start start 2023-03-03T06:26:41.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18e1928700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18dc156ba0 0x7f18dc071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:41.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18e1928700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18dc15d6e0 0x7f18dc0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:41.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18e1928700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18dc0cf3b0 con 0x7f18dc0cfd80 2023-03-03T06:26:41.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18e1928700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18dc156f70 con 0x7f18dc156ba0 2023-03-03T06:26:41.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18e1928700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18dc0cd320 con 0x7f18dc15d6e0 2023-03-03T06:26:41.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18e0926700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18dc156ba0 0x7f18dc071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:41.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18e0926700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18dc156ba0 0x7f18dc071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57508/0 (socket says 172.21.15.3:57508) 2023-03-03T06:26:41.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18e0926700 1 -- 172.21.15.3:0/3817792090 learned_addr learned my addr 172.21.15.3:0/3817792090 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:41.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18d3fff700 1 --2- 172.21.15.3:0/3817792090 >> v2:172.21.15.32:3300/0 conn(0x7f18dc15d6e0 0x7f18dc0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:41.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18e0926700 1 -- 172.21.15.3:0/3817792090 >> v2:172.21.15.32:3300/0 conn(0x7f18dc15d6e0 msgr2=0x7f18dc0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:41.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18e0926700 1 --2- 172.21.15.3:0/3817792090 >> v2:172.21.15.32:3300/0 conn(0x7f18dc15d6e0 0x7f18dc0cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:41.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18e0926700 1 -- 172.21.15.3:0/3817792090 >> v1:172.21.15.32:6789/0 conn(0x7f18dc0cfd80 legacy=0x7f18dc0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:41.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18e0926700 1 -- 172.21.15.3:0/3817792090 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc0cd600 con 0x7f18dc156ba0 2023-03-03T06:26:41.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.262+0000 7f18e0926700 1 --2- 172.21.15.3:0/3817792090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18dc156ba0 0x7f18dc071280 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f18dc0b1ca0 tx=0x7f18d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999118a723913ffc server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:41.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.266+0000 7f18d37fe700 1 -- 172.21.15.3:0/3817792090 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d800e070 con 0x7f18dc156ba0 2023-03-03T06:26:41.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.266+0000 7f18d37fe700 1 -- 172.21.15.3:0/3817792090 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d800b910 con 0x7f18dc156ba0 2023-03-03T06:26:41.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.266+0000 7f18d37fe700 1 -- 172.21.15.3:0/3817792090 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d800bcc0 con 0x7f18dc156ba0 2023-03-03T06:26:41.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.266+0000 7f18e1928700 1 -- 172.21.15.3:0/3817792090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18dc156ba0 msgr2=0x7f18dc071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:41.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.266+0000 7f18e1928700 1 --2- 172.21.15.3:0/3817792090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18dc156ba0 0x7f18dc071280 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f18dc0b1ca0 tx=0x7f18d800b330 comp rx=0 tx=0).stop 2023-03-03T06:26:41.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.266+0000 7f18e1928700 1 -- 172.21.15.3:0/3817792090 shutdown_connections 2023-03-03T06:26:41.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.266+0000 7f18e1928700 1 --2- 172.21.15.3:0/3817792090 >> v2:172.21.15.32:3300/0 conn(0x7f18dc15d6e0 0x7f18dc0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:41.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.266+0000 7f18e1928700 1 --2- 172.21.15.3:0/3817792090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18dc156ba0 0x7f18dc071280 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:41.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.266+0000 7f18e1928700 1 -- 172.21.15.3:0/3817792090 >> 172.21.15.3:0/3817792090 conn(0x7f18dc06e5c0 msgr2=0x7f18dc06e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:41.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.266+0000 7f18e1928700 1 -- 172.21.15.3:0/3817792090 shutdown_connections 2023-03-03T06:26:41.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.266+0000 7f18e1928700 1 -- 172.21.15.3:0/3817792090 wait complete. 2023-03-03T06:26:41.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.270+0000 7f18e1928700 1 Processor -- start 2023-03-03T06:26:41.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.270+0000 7f18e1928700 1 -- start start 2023-03-03T06:26:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.270+0000 7f18e1928700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18dc0cfd80 0x7f18dc14e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.270+0000 7f18e1928700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc156ba0 0x7f18dc0c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.270+0000 7f18e1928700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18dc15d6e0 0x7f18dc15fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.270+0000 7f18e1928700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0b7a00 con 0x7f18dc15d6e0 2023-03-03T06:26:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.270+0000 7f18e1928700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0b6e10 con 0x7f18dc156ba0 2023-03-03T06:26:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.270+0000 7f18e1928700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc14e630 con 0x7f18dc0cfd80 2023-03-03T06:26:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18e1127700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18dc15d6e0 0x7f18dc15fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18e1127700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18dc15d6e0 0x7f18dc15fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57514/0 (socket says 172.21.15.3:57514) 2023-03-03T06:26:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18d3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc156ba0 0x7f18dc0c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18e1127700 1 -- 172.21.15.3:0/4137000007 learned_addr learned my addr 172.21.15.3:0/4137000007 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18e0926700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18dc0cfd80 0x7f18dc14e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:41.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18e1127700 1 -- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18dc0cfd80 msgr2=0x7f18dc14e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18e1127700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18dc0cfd80 0x7f18dc14e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18e1127700 1 -- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc156ba0 msgr2=0x7f18dc0c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18e1127700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc156ba0 0x7f18dc0c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18e1127700 1 -- 172.21.15.3:0/4137000007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d8009b30 con 0x7f18dc15d6e0 2023-03-03T06:26:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18d3fff700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc156ba0 0x7f18dc0c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:41.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18e1127700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18dc15d6e0 0x7f18dc15fd20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f18d400c460 tx=0x7f18d400c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:41.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18d17fa700 1 -- 172.21.15.3:0/4137000007 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d40052d0 con 0x7f18dc15d6e0 2023-03-03T06:26:41.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18d17fa700 1 -- 172.21.15.3:0/4137000007 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d4005430 con 0x7f18dc15d6e0 2023-03-03T06:26:41.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18d17fa700 1 -- 172.21.15.3:0/4137000007 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d40055a0 con 0x7f18dc15d6e0 2023-03-03T06:26:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18e1928700 1 -- 172.21.15.3:0/4137000007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc0717c0 con 0x7f18dc15d6e0 2023-03-03T06:26:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.274+0000 7f18e1928700 1 -- 172.21.15.3:0/4137000007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc0cd8e0 con 0x7f18dc15d6e0 2023-03-03T06:26:41.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.278+0000 7f18e1928700 1 -- 172.21.15.3:0/4137000007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc0cef40 con 0x7f18dc15d6e0 2023-03-03T06:26:41.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.278+0000 7f18d17fa700 1 -- 172.21.15.3:0/4137000007 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f18d4019030 con 0x7f18dc15d6e0 2023-03-03T06:26:41.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.282+0000 7f18d17fa700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f18c8064ee0 0x7f18c8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:41.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.282+0000 7f18d17fa700 1 -- 172.21.15.3:0/4137000007 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6047+0+0 (secure 0 0 0) 0x7f18d40810c0 con 0x7f18dc15d6e0 2023-03-03T06:26:41.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.282+0000 7f18e0926700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f18c8064ee0 0x7f18c8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:41.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.282+0000 7f18e0926700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f18c8064ee0 0x7f18c8067390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f18d8000f80 tx=0x7f18d8009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:41.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.286+0000 7f18d17fa700 1 -- 172.21.15.3:0/4137000007 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f18d404f180 con 0x7f18dc15d6e0 2023-03-03T06:26:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:41.462+0000 7f18e1928700 1 -- 172.21.15.3:0/4137000007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f18dc0b7b40 con 0x7f18dc15d6e0 2023-03-03T06:26:42.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:42.010+0000 7f18d17fa700 1 -- 172.21.15.3:0/4137000007 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v34) v1 ==== 178+0+0 (secure 0 0 0) 0x7f18dc0b7b40 con 0x7f18dc15d6e0 2023-03-03T06:26:42.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:42.018+0000 7f18e1928700 1 -- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f18c8064ee0 msgr2=0x7f18c8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:42.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:42.018+0000 7f18e1928700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f18c8064ee0 0x7f18c8067390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f18d8000f80 tx=0x7f18d8009000 comp rx=0 tx=0).stop 2023-03-03T06:26:42.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:42.018+0000 7f18e1928700 1 -- 172.21.15.3:0/4137000007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18dc15d6e0 msgr2=0x7f18dc15fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:42.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:42.018+0000 7f18e1928700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18dc15d6e0 0x7f18dc15fd20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f18d400c460 tx=0x7f18d400c770 comp rx=0 tx=0).stop 2023-03-03T06:26:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:42.018+0000 7f18e1928700 1 -- 172.21.15.3:0/4137000007 shutdown_connections 2023-03-03T06:26:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:42.018+0000 7f18e1928700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f18c8064ee0 0x7f18c8067390 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:42.018+0000 7f18e1928700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18dc15d6e0 0x7f18dc15fd20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:42.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:42.018+0000 7f18e1928700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc156ba0 0x7f18dc0c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:42.018+0000 7f18e1928700 1 --2- 172.21.15.3:0/4137000007 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f18dc0cfd80 0x7f18dc14e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:42.018+0000 7f18e1928700 1 -- 172.21.15.3:0/4137000007 >> 172.21.15.3:0/4137000007 conn(0x7f18dc06e5c0 msgr2=0x7f18dc15ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:42.018+0000 7f18e1928700 1 -- 172.21.15.3:0/4137000007 shutdown_connections 2023-03-03T06:26:42.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:42.018+0000 7f18e1928700 1 -- 172.21.15.3:0/4137000007 wait complete. 2023-03-03T06:26:42.027 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data_ec' removed 2023-03-03T06:26:42.059 INFO:tasks.ceph.mds.a:Restarting daemon 2023-03-03T06:26:42.059 INFO:tasks.ceph.mds.a:Stopping old one... 2023-03-03T06:26:42.059 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-03-03T06:26:42.060 INFO:teuthology.orchestra.run:waiting for 300 2023-03-03T06:26:42.060 INFO:tasks.ceph.mds.c:Restarting daemon 2023-03-03T06:26:42.060 INFO:tasks.ceph.mds.c:Stopping old one... 2023-03-03T06:26:42.060 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-03-03T06:26:42.060 INFO:teuthology.orchestra.run:waiting for 300 2023-03-03T06:26:42.061 INFO:tasks.ceph.mds.b:Restarting daemon 2023-03-03T06:26:42.061 INFO:tasks.ceph.mds.b:Stopping old one... 2023-03-03T06:26:42.061 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-03-03T06:26:42.061 INFO:teuthology.orchestra.run:waiting for 300 2023-03-03T06:26:42.062 INFO:tasks.ceph.mds.d:Restarting daemon 2023-03-03T06:26:42.062 INFO:tasks.ceph.mds.d:Stopping old one... 2023-03-03T06:26:42.062 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-03-03T06:26:42.062 INFO:teuthology.orchestra.run:waiting for 300 2023-03-03T06:26:42.090 INFO:tasks.ceph.mds.d:Stopped 2023-03-03T06:26:42.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-03-03T06:26:42.092 INFO:tasks.ceph.mds.b:Stopped 2023-03-03T06:26:42.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-03-03T06:26:42.094 INFO:tasks.ceph.mds.c:Stopped 2023-03-03T06:26:42.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-03-03T06:26:42.139 INFO:tasks.ceph.mds.d:Started 2023-03-03T06:26:42.140 DEBUG:teuthology.parallel:result is None 2023-03-03T06:26:42.142 INFO:tasks.ceph.mds.b:Started 2023-03-03T06:26:42.142 DEBUG:teuthology.parallel:result is None 2023-03-03T06:26:42.150 INFO:tasks.ceph.mds.c:Started 2023-03-03T06:26:42.150 DEBUG:teuthology.parallel:result is None 2023-03-03T06:26:42.287 INFO:tasks.ceph.mds.d.smithi032.stdout:starting mds.d at 2023-03-03T06:26:42.287 INFO:tasks.ceph.mds.b.smithi032.stdout:starting mds.b at 2023-03-03T06:26:42.288 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2023-03-03T06:26:45.785 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-03-03T06:26:48.095 INFO:tasks.ceph.mds.a:Stopped 2023-03-03T06:26:48.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-03-03T06:26:48.104 INFO:tasks.ceph.mds.a:Started 2023-03-03T06:26:48.104 DEBUG:teuthology.parallel:result is None 2023-03-03T06:26:48.105 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-03-03T06:26:48.250 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2023-03-03T06:26:48.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.298+0000 7fe7f56c4700 1 Processor -- start 2023-03-03T06:26:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.298+0000 7fe7f56c4700 1 -- start start 2023-03-03T06:26:48.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.302+0000 7fe7f56c4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7f015c750 0x7fe7f0163370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:48.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.302+0000 7fe7f56c4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f00cdd30 con 0x7fe7f0158440 2023-03-03T06:26:48.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.302+0000 7fe7f56c4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f00ca4d0 con 0x7fe7f00aee70 2023-03-03T06:26:48.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.302+0000 7fe7f56c4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f00cd6e0 con 0x7fe7f015c750 2023-03-03T06:26:48.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.306+0000 7fe7effff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe7f0158440 0x7fe7f00c9a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45914/0 (socket says 172.21.15.3:45914) 2023-03-03T06:26:48.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.306+0000 7fe7effff700 1 -- 172.21.15.3:0/2413058371 learned_addr learned my addr 172.21.15.3:0/2413058371 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:48.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.306+0000 7fe7f4ec3700 1 --2- 172.21.15.3:0/2413058371 >> v2:172.21.15.32:3300/0 conn(0x7fe7f015c750 0x7fe7f0163370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:48.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.306+0000 7fe7eeffd700 1 -- 172.21.15.3:0/2413058371 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901430707 0 0) 0x7fe7f00cdd30 con 0x7fe7f0158440 2023-03-03T06:26:48.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.306+0000 7fe7eeffd700 1 -- 172.21.15.3:0/2413058371 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7d80034f0 con 0x7fe7f0158440 2023-03-03T06:26:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.306+0000 7fe7eeffd700 1 -- 172.21.15.3:0/2413058371 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686989789 0 0) 0x7fe7f00ca4d0 con 0x7fe7f00aee70 2023-03-03T06:26:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.306+0000 7fe7f4ec3700 1 -- 172.21.15.3:0/2413058371 >> v1:172.21.15.32:6789/0 conn(0x7fe7f00aee70 legacy=0x7fe7f015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.306+0000 7fe7f4ec3700 1 -- 172.21.15.3:0/2413058371 >> v1:172.21.15.3:6789/0 conn(0x7fe7f0158440 legacy=0x7fe7f00c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.306+0000 7fe7f4ec3700 1 -- 172.21.15.3:0/2413058371 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f00cd9c0 con 0x7fe7f015c750 2023-03-03T06:26:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.306+0000 7fe7f4ec3700 1 --2- 172.21.15.3:0/2413058371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f015c750 0x7fe7f0163370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009d90 tx=0x7fe7e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49c56f1edaf04e3b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:48.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.306+0000 7fe7eeffd700 1 -- 172.21.15.3:0/2413058371 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e800e070 con 0x7fe7f015c750 2023-03-03T06:26:48.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.306+0000 7fe7eeffd700 1 -- 172.21.15.3:0/2413058371 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e800bb20 con 0x7fe7f015c750 2023-03-03T06:26:48.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.306+0000 7fe7eeffd700 1 -- 172.21.15.3:0/2413058371 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e800bed0 con 0x7fe7f015c750 2023-03-03T06:26:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.310+0000 7fe7f56c4700 1 -- 172.21.15.3:0/2413058371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f015c750 msgr2=0x7fe7f0163370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.310+0000 7fe7f56c4700 1 --2- 172.21.15.3:0/2413058371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f015c750 0x7fe7f0163370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009d90 tx=0x7fe7e800b330 comp rx=0 tx=0).stop 2023-03-03T06:26:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.310+0000 7fe7f56c4700 1 -- 172.21.15.3:0/2413058371 shutdown_connections 2023-03-03T06:26:48.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.310+0000 7fe7f56c4700 1 --2- 172.21.15.3:0/2413058371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f015c750 0x7fe7f0163370 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:48.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.310+0000 7fe7f56c4700 1 -- 172.21.15.3:0/2413058371 >> 172.21.15.3:0/2413058371 conn(0x7fe7f0070a30 msgr2=0x7fe7f0070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:48.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.310+0000 7fe7f56c4700 1 -- 172.21.15.3:0/2413058371 shutdown_connections 2023-03-03T06:26:48.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.310+0000 7fe7f56c4700 1 -- 172.21.15.3:0/2413058371 wait complete. 2023-03-03T06:26:48.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.314+0000 7fe7f56c4700 1 Processor -- start 2023-03-03T06:26:48.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.314+0000 7fe7f56c4700 1 -- start start 2023-03-03T06:26:48.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.314+0000 7fe7f56c4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7f00aee70 0x7fe7f01edf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:48.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.314+0000 7fe7f56c4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f0158440 0x7fe7f01ee980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:48.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.314+0000 7fe7f56c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f015c750 0x7fe7f00b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.314+0000 7fe7f56c4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f0148510 con 0x7fe7f00aee70 2023-03-03T06:26:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.314+0000 7fe7f56c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f01ee4b0 con 0x7fe7f015c750 2023-03-03T06:26:48.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.314+0000 7fe7f56c4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f01ee750 con 0x7fe7f0158440 2023-03-03T06:26:48.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.314+0000 7fe7effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7f00aee70 0x7fe7f01edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7effff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7f00aee70 0x7fe7f01edf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57556/0 (socket says 172.21.15.3:57556) 2023-03-03T06:26:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7effff700 1 -- 172.21.15.3:0/3496234511 learned_addr learned my addr 172.21.15.3:0/3496234511 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7ef7fe700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f0158440 0x7fe7f01ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7f4ec3700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f015c750 0x7fe7f00b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:48.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7effff700 1 -- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f0158440 msgr2=0x7fe7f01ee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:48.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7effff700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f0158440 0x7fe7f01ee980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:48.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7effff700 1 -- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f015c750 msgr2=0x7fe7f00b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:48.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7effff700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f015c750 0x7fe7f00b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:48.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7effff700 1 -- 172.21.15.3:0/3496234511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e8009a70 con 0x7fe7f00aee70 2023-03-03T06:26:48.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7effff700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7f00aee70 0x7fe7f01edf70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000c4f0 tx=0x7fe7e000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7ecff9700 1 -- 172.21.15.3:0/3496234511 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0005d90 con 0x7fe7f00aee70 2023-03-03T06:26:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7ecff9700 1 -- 172.21.15.3:0/3496234511 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e00070d0 con 0x7fe7f00aee70 2023-03-03T06:26:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7ecff9700 1 -- 172.21.15.3:0/3496234511 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0007440 con 0x7fe7f00aee70 2023-03-03T06:26:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7f56c4700 1 -- 172.21.15.3:0/3496234511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f015cd90 con 0x7fe7f00aee70 2023-03-03T06:26:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.318+0000 7fe7f56c4700 1 -- 172.21.15.3:0/3496234511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f00ca7b0 con 0x7fe7f00aee70 2023-03-03T06:26:48.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.322+0000 7fe7f56c4700 1 -- 172.21.15.3:0/3496234511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f00ca120 con 0x7fe7f00aee70 2023-03-03T06:26:48.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.322+0000 7fe7ecff9700 1 -- 172.21.15.3:0/3496234511 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe7e0019030 con 0x7fe7f00aee70 2023-03-03T06:26:48.330 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.326+0000 7fe7ecff9700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe7dc064ee0 0x7fe7dc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:48.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.326+0000 7fe7ecff9700 1 -- 172.21.15.3:0/3496234511 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fe7e0080f50 con 0x7fe7f00aee70 2023-03-03T06:26:48.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.326+0000 7fe7ef7fe700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe7dc064ee0 0x7fe7dc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:48.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.326+0000 7fe7ef7fe700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe7dc064ee0 0x7fe7dc067390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00b3860 tx=0x7fe7e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:48.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.330+0000 7fe7ecff9700 1 -- 172.21.15.3:0/3496234511 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe7e004a400 con 0x7fe7f00aee70 2023-03-03T06:26:48.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:48.518+0000 7fe7f56c4700 1 -- 172.21.15.3:0/3496234511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe7f00ae770 con 0x7fe7f00aee70 2023-03-03T06:26:49.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.286+0000 7fe7ecff9700 1 -- 172.21.15.3:0/3496234511 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe7e0017080 con 0x7fe7f00aee70 2023-03-03T06:26:49.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.290+0000 7fe7f56c4700 1 -- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe7dc064ee0 msgr2=0x7fe7dc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.290+0000 7fe7f56c4700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe7dc064ee0 0x7fe7dc067390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00b3860 tx=0x7fe7e4009040 comp rx=0 tx=0).stop 2023-03-03T06:26:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.290+0000 7fe7f56c4700 1 -- 172.21.15.3:0/3496234511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7f00aee70 msgr2=0x7fe7f01edf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.290+0000 7fe7f56c4700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7f00aee70 0x7fe7f01edf70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000c4f0 tx=0x7fe7e000c800 comp rx=0 tx=0).stop 2023-03-03T06:26:49.294 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.290+0000 7fe7f56c4700 1 -- 172.21.15.3:0/3496234511 shutdown_connections 2023-03-03T06:26:49.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.290+0000 7fe7f56c4700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe7dc064ee0 0x7fe7dc067390 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:49.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.290+0000 7fe7f56c4700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7f015c750 0x7fe7f00b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:49.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.290+0000 7fe7f56c4700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe7f0158440 0x7fe7f01ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:49.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.290+0000 7fe7f56c4700 1 --2- 172.21.15.3:0/3496234511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7f00aee70 0x7fe7f01edf70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:49.295 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.290+0000 7fe7f56c4700 1 -- 172.21.15.3:0/3496234511 >> 172.21.15.3:0/3496234511 conn(0x7fe7f0070a30 msgr2=0x7fe7f0148d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:49.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.290+0000 7fe7f56c4700 1 -- 172.21.15.3:0/3496234511 shutdown_connections 2023-03-03T06:26:49.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.290+0000 7fe7f56c4700 1 -- 172.21.15.3:0/3496234511 wait complete. 2023-03-03T06:26:49.299 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2023-03-03T06:26:49.334 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-03-03T06:26:49.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.546+0000 7f6841828700 1 Processor -- start 2023-03-03T06:26:49.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.546+0000 7f6841828700 1 -- start start 2023-03-03T06:26:49.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6841828700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f683c06a780 0x7f683c158440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:49.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6841828700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c0aee70 0x7f683c0c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:49.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6841828700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c0cdd30 con 0x7f683c0c9f50 2023-03-03T06:26:49.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6841828700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683c158980 con 0x7f683c06a780 2023-03-03T06:26:49.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6841828700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c0ca4d0 con 0x7f683c0aee70 2023-03-03T06:26:49.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6841027700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f683c0c9f50 0x7f683c15c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45924/0 (socket says 172.21.15.3:45924) 2023-03-03T06:26:49.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6840826700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f683c06a780 0x7f683c158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:49.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6841027700 1 -- 172.21.15.3:0/2387988799 learned_addr learned my addr 172.21.15.3:0/2387988799 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6833fff700 1 --2- 172.21.15.3:0/2387988799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c0aee70 0x7f683c0c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6840826700 1 -- 172.21.15.3:0/2387988799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c0aee70 msgr2=0x7f683c0c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6840826700 1 --2- 172.21.15.3:0/2387988799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c0aee70 0x7f683c0c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6840826700 1 -- 172.21.15.3:0/2387988799 >> v1:172.21.15.3:6789/0 conn(0x7f683c0c9f50 legacy=0x7f683c15c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:49.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6840826700 1 -- 172.21.15.3:0/2387988799 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c0cd6e0 con 0x7f683c06a780 2023-03-03T06:26:49.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f68337fe700 1 -- 172.21.15.3:0/2387988799 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329881817 0 0) 0x7f683c0cdd30 con 0x7f683c0c9f50 2023-03-03T06:26:49.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.550+0000 7f6840826700 1 --2- 172.21.15.3:0/2387988799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c06a780 0x7f683c158440 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6838009880 tx=0x7f6838009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67ceede4e58ea5ec server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:49.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.554+0000 7f68337fe700 1 -- 172.21.15.3:0/2387988799 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838011830 con 0x7f683c06a780 2023-03-03T06:26:49.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.554+0000 7f68337fe700 1 -- 172.21.15.3:0/2387988799 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6838011990 con 0x7f683c06a780 2023-03-03T06:26:49.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.554+0000 7f68337fe700 1 -- 172.21.15.3:0/2387988799 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838011d20 con 0x7f683c06a780 2023-03-03T06:26:49.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.554+0000 7f6841828700 1 -- 172.21.15.3:0/2387988799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c06a780 msgr2=0x7f683c158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:49.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.554+0000 7f6841828700 1 --2- 172.21.15.3:0/2387988799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c06a780 0x7f683c158440 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6838009880 tx=0x7f6838009b90 comp rx=0 tx=0).stop 2023-03-03T06:26:49.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.554+0000 7f6841828700 1 -- 172.21.15.3:0/2387988799 shutdown_connections 2023-03-03T06:26:49.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.554+0000 7f6841828700 1 --2- 172.21.15.3:0/2387988799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c0aee70 0x7f683c0c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:49.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.554+0000 7f6841828700 1 --2- 172.21.15.3:0/2387988799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c06a780 0x7f683c158440 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:49.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.554+0000 7f6841828700 1 -- 172.21.15.3:0/2387988799 >> 172.21.15.3:0/2387988799 conn(0x7f683c070a30 msgr2=0x7f683c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:49.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.554+0000 7f6841828700 1 -- 172.21.15.3:0/2387988799 shutdown_connections 2023-03-03T06:26:49.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.554+0000 7f6841828700 1 -- 172.21.15.3:0/2387988799 wait complete. 2023-03-03T06:26:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.554+0000 7f6841828700 1 Processor -- start 2023-03-03T06:26:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6841828700 1 -- start start 2023-03-03T06:26:49.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6841828700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c06a780 0x7f683c14cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:49.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6841828700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c0aee70 0x7f683c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6841828700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f683c0c9f50 0x7f683c06d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6841828700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c152250 con 0x7f683c0c9f50 2023-03-03T06:26:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6841828700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c1524d0 con 0x7f683c06a780 2023-03-03T06:26:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6841828700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c0b2280 con 0x7f683c0aee70 2023-03-03T06:26:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6833fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c0aee70 0x7f683c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6841027700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f683c0c9f50 0x7f683c06d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6833fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c0aee70 0x7f683c06cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:38592/0 (socket says 172.21.15.3:38592) 2023-03-03T06:26:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6833fff700 1 -- 172.21.15.3:0/3339051235 learned_addr learned my addr 172.21.15.3:0/3339051235 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6840826700 1 --2- 172.21.15.3:0/3339051235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c06a780 0x7f683c14cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:49.565 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6841027700 1 -- 172.21.15.3:0/3339051235 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c0aee70 msgr2=0x7f683c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:49.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6841027700 1 --2- 172.21.15.3:0/3339051235 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c0aee70 0x7f683c06cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:49.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6841027700 1 -- 172.21.15.3:0/3339051235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c06a780 msgr2=0x7f683c14cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:49.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6841027700 1 --2- 172.21.15.3:0/3339051235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c06a780 0x7f683c14cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:49.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.558+0000 7f6841027700 1 -- 172.21.15.3:0/3339051235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68380094b0 con 0x7f683c0c9f50 2023-03-03T06:26:49.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.562+0000 7f6841027700 1 --2- 172.21.15.3:0/3339051235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f683c0c9f50 0x7f683c06d240 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f683400caf0 tx=0x7f683400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:49.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.562+0000 7f68317fa700 1 -- 172.21.15.3:0/3339051235 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683400a9e0 con 0x7f683c0c9f50 2023-03-03T06:26:49.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.562+0000 7f68317fa700 1 -- 172.21.15.3:0/3339051235 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683400ab40 con 0x7f683c0c9f50 2023-03-03T06:26:49.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.562+0000 7f6841828700 1 -- 172.21.15.3:0/3339051235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c0ca7b0 con 0x7f683c0c9f50 2023-03-03T06:26:49.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.562+0000 7f68317fa700 1 -- 172.21.15.3:0/3339051235 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68340070d0 con 0x7f683c0c9f50 2023-03-03T06:26:49.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.562+0000 7f6841828700 1 -- 172.21.15.3:0/3339051235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c0cd9c0 con 0x7f683c0c9f50 2023-03-03T06:26:49.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.562+0000 7f6841828700 1 -- 172.21.15.3:0/3339051235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683c15cd90 con 0x7f683c0c9f50 2023-03-03T06:26:49.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.566+0000 7f68317fa700 1 -- 172.21.15.3:0/3339051235 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6834010070 con 0x7f683c0c9f50 2023-03-03T06:26:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.566+0000 7f68317fa700 1 --2- 172.21.15.3:0/3339051235 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6828064ee0 0x7f6828067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:49.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.566+0000 7f68317fa700 1 -- 172.21.15.3:0/3339051235 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6834081580 con 0x7f683c0c9f50 2023-03-03T06:26:49.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.566+0000 7f6840826700 1 --2- 172.21.15.3:0/3339051235 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6828064ee0 0x7f6828067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:49.575 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.570+0000 7f6840826700 1 --2- 172.21.15.3:0/3339051235 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6828064ee0 0x7f6828067390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68380090b0 tx=0x7f6838008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.574+0000 7f68317fa700 1 -- 172.21.15.3:0/3339051235 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f683404f750 con 0x7f683c0c9f50 2023-03-03T06:26:49.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.814+0000 7f6841828700 1 -- 172.21.15.3:0/3339051235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f683c073e90 con 0x7f683c0c9f50 2023-03-03T06:26:49.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.814+0000 7f68317fa700 1 -- 172.21.15.3:0/3339051235 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v7) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f6834018020 con 0x7f683c0c9f50 2023-03-03T06:26:49.821 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:49.822 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-03-03T06:26:49.822 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2023-03-03T06:26:49.822 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.822 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2023-03-03T06:26:49.822 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCAkgFkfHvwJhAAPEj36Q5cJyrlQYjEaG2sGg==", 2023-03-03T06:26:49.822 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.822 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-03-03T06:26:49.823 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-03-03T06:26:49.823 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-03-03T06:26:49.823 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.823 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.823 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.823 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.823 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2023-03-03T06:26:49.824 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCBkgFkTOo9BBAA78PiqKJ59InZKdWZFMCN7w==", 2023-03-03T06:26:49.824 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.824 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-03-03T06:26:49.824 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-03-03T06:26:49.824 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-03-03T06:26:49.824 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.824 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.824 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.825 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.825 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2023-03-03T06:26:49.825 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCAkgFkGgRnNBAA5+REl/roeIhnxFvAmB5okw==", 2023-03-03T06:26:49.825 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.825 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-03-03T06:26:49.825 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-03-03T06:26:49.825 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-03-03T06:26:49.826 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.826 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.826 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.826 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.826 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2023-03-03T06:26:49.826 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCBkgFkRmqnERAAd8kObftCMq5QZDGFzcbtWw==", 2023-03-03T06:26:49.826 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.826 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-03-03T06:26:49.827 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-03-03T06:26:49.827 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-03-03T06:26:49.827 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.827 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.827 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.827 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.827 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2023-03-03T06:26:49.827 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCFkgFkuEucExAA3ukOiXqZpzCZqHrdMkpr7w==", 2023-03-03T06:26:49.828 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.828 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-03-03T06:26:49.828 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-03-03T06:26:49.828 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.828 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.828 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.828 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.828 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2023-03-03T06:26:49.829 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCIkgFkAWd2NhAA3X8Z7xuVlRU2lYChQJVWKg==", 2023-03-03T06:26:49.829 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.829 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-03-03T06:26:49.829 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-03-03T06:26:49.829 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.829 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.829 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.830 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.830 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2023-03-03T06:26:49.830 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCMkgFkbIQTHRAAyyfQ1Qc3LkCwNrOYWhM3DA==", 2023-03-03T06:26:49.830 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.830 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-03-03T06:26:49.830 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-03-03T06:26:49.830 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.830 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.831 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.831 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.831 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2023-03-03T06:26:49.831 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCQkgFkQhK8AxAAm58cosvi5WKSGo+ez131gg==", 2023-03-03T06:26:49.831 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.831 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-03-03T06:26:49.831 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-03-03T06:26:49.831 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.832 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.832 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.832 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.832 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2023-03-03T06:26:49.832 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCUkgFkmRKDNxAABkz6ksqRui2bySWPQQfBEg==", 2023-03-03T06:26:49.832 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.832 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-03-03T06:26:49.832 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-03-03T06:26:49.833 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.833 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.833 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.833 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.833 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2023-03-03T06:26:49.833 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCYkgFkIhe8HBAAyvuIJoh5EB4nGj6S443d1A==", 2023-03-03T06:26:49.833 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.833 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-03-03T06:26:49.834 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-03-03T06:26:49.834 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.834 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.834 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.834 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.834 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2023-03-03T06:26:49.834 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCckgFkEOgvAhAAcQBWJ6xLJ4JY8KDnosIRGQ==", 2023-03-03T06:26:49.835 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.835 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-03-03T06:26:49.835 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-03-03T06:26:49.835 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.835 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.835 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.835 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.835 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2023-03-03T06:26:49.836 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCfkgFk+yavJBAAEDFnVa1ZIFMR8X9asFniNA==", 2023-03-03T06:26:49.836 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.836 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2023-03-03T06:26:49.836 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2023-03-03T06:26:49.836 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.836 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.837 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.837 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2023-03-03T06:26:49.837 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCBkgFkq25UGhAAPQ0Adr+W73H75r08iuEYDQ==", 2023-03-03T06:26:49.837 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.837 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-03-03T06:26:49.837 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-03-03T06:26:49.837 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-03-03T06:26:49.837 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-03-03T06:26:49.838 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.838 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.838 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.838 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2023-03-03T06:26:49.838 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCBkgFkPYhfKhAAPtcpq/S9ReAFseRm31sPzg==", 2023-03-03T06:26:49.838 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.838 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-03-03T06:26:49.838 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-03-03T06:26:49.839 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-03-03T06:26:49.839 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-03-03T06:26:49.839 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.839 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.839 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.839 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2023-03-03T06:26:49.839 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCBkgFkgHYgIhAAKg+r8B3OOYp8KU9Oly+U1Q==", 2023-03-03T06:26:49.839 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.840 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-03-03T06:26:49.840 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-03-03T06:26:49.840 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-03-03T06:26:49.840 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-03-03T06:26:49.840 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.840 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.840 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.840 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2023-03-03T06:26:49.841 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCBkgFkPL17MhAAKe5MQgwKkfsfuhe4Q49STg==", 2023-03-03T06:26:49.841 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.841 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2023-03-03T06:26:49.841 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2023-03-03T06:26:49.841 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2023-03-03T06:26:49.841 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2023-03-03T06:26:49.841 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.842 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.842 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.842 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2023-03-03T06:26:49.842 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQB/kgFk3s+LLhAAFOxHi6w3unwdUWaOQkISYQ==", 2023-03-03T06:26:49.842 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.842 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-03-03T06:26:49.842 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2023-03-03T06:26:49.842 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2023-03-03T06:26:49.843 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.843 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.843 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.843 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.843 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mds", 2023-03-03T06:26:49.843 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCskgFk8F68MxAAAgEifwLXXMTEDlOqIAWYqQ==", 2023-03-03T06:26:49.843 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.843 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mds" 2023-03-03T06:26:49.844 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.844 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.844 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.844 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mgr", 2023-03-03T06:26:49.844 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCskgFk+W+8MxAAtiwJU+GpK+9HoEHgulFiNA==", 2023-03-03T06:26:49.844 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.844 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mgr" 2023-03-03T06:26:49.844 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.845 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.845 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-osd", 2023-03-03T06:26:49.845 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCskgFku4C8MxAAGnRdCru9mwC0OFBtDNpHcQ==", 2023-03-03T06:26:49.845 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.845 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-osd" 2023-03-03T06:26:49.845 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.846 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.846 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd", 2023-03-03T06:26:49.846 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCskgFky4+8MxAAiMyd0SJEJx7/is4CrInTnw==", 2023-03-03T06:26:49.846 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.846 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd" 2023-03-03T06:26:49.846 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.847 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.847 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-03-03T06:26:49.847 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCskgFk2p+8MxAAP6WLGP3b0eBlo/vD9I+0og==", 2023-03-03T06:26:49.847 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.847 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-03-03T06:26:49.847 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.847 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.847 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.848 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rgw", 2023-03-03T06:26:49.848 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCskgFkC7O8MxAAEc8rYVnP53uAk5/c8cmiZQ==", 2023-03-03T06:26:49.848 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.848 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rgw" 2023-03-03T06:26:49.848 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.848 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.848 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.848 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2023-03-03T06:26:49.849 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCAkgFkNkJqHRAACPVn302t25/DzryFbFr12w==", 2023-03-03T06:26:49.849 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.849 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-03-03T06:26:49.849 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-03-03T06:26:49.849 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.849 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.849 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:49.849 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:49.850 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2023-03-03T06:26:49.850 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQCAkgFkjLAIFRAA5hYq+AKWH11R3giPKunqog==", 2023-03-03T06:26:49.850 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2023-03-03T06:26:49.850 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2023-03-03T06:26:49.850 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2023-03-03T06:26:49.850 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2023-03-03T06:26:49.850 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.850 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:49.851 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2023-03-03T06:26:49.851 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-03-03T06:26:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.818+0000 7f6841828700 1 -- 172.21.15.3:0/3339051235 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6828064ee0 msgr2=0x7f6828067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.818+0000 7f6841828700 1 --2- 172.21.15.3:0/3339051235 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6828064ee0 0x7f6828067390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68380090b0 tx=0x7f6838008030 comp rx=0 tx=0).stop 2023-03-03T06:26:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.818+0000 7f6841828700 1 -- 172.21.15.3:0/3339051235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f683c0c9f50 msgr2=0x7f683c06d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.818+0000 7f6841828700 1 --2- 172.21.15.3:0/3339051235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f683c0c9f50 0x7f683c06d240 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f683400caf0 tx=0x7f683400ce00 comp rx=0 tx=0).stop 2023-03-03T06:26:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.822+0000 7f6841828700 1 -- 172.21.15.3:0/3339051235 shutdown_connections 2023-03-03T06:26:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.822+0000 7f6841828700 1 --2- 172.21.15.3:0/3339051235 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6828064ee0 0x7f6828067390 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.822+0000 7f6841828700 1 --2- 172.21.15.3:0/3339051235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f683c0c9f50 0x7f683c06d240 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.822+0000 7f6841828700 1 --2- 172.21.15.3:0/3339051235 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f683c0aee70 0x7f683c06cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.822+0000 7f6841828700 1 --2- 172.21.15.3:0/3339051235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f683c06a780 0x7f683c14cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.822+0000 7f6841828700 1 -- 172.21.15.3:0/3339051235 >> 172.21.15.3:0/3339051235 conn(0x7f683c070a30 msgr2=0x7f683c0cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.822+0000 7f6841828700 1 -- 172.21.15.3:0/3339051235 shutdown_connections 2023-03-03T06:26:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:49.822+0000 7f6841828700 1 -- 172.21.15.3:0/3339051235 wait complete. 2023-03-03T06:26:49.862 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-03-03T06:26:50.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.066+0000 7fd6288a6700 1 Processor -- start 2023-03-03T06:26:50.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.066+0000 7fd6288a6700 1 -- start start 2023-03-03T06:26:50.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd6288a6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240cfd30 0x7fd6240d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:50.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd6288a6700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd624160440 con 0x7fd624153670 2023-03-03T06:26:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd6288a6700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6240cd3e0 con 0x7fd6241608d0 2023-03-03T06:26:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd6288a6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240cd6c0 con 0x7fd6240cfd30 2023-03-03T06:26:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd622d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd624153670 0x7fd624071350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45926/0 (socket says 172.21.15.3:45926) 2023-03-03T06:26:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd622d9d700 1 -- 172.21.15.3:0/1655788960 learned_addr learned my addr 172.21.15.3:0/1655788960 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:50.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd62359e700 1 --2- 172.21.15.3:0/1655788960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240cfd30 0x7fd6240d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:50.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643603991 0 0) 0x7fd624160440 con 0x7fd624153670 2023-03-03T06:26:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd60c0035d0 con 0x7fd624153670 2023-03-03T06:26:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2124506708 0 0) 0x7fd60c0035d0 con 0x7fd624153670 2023-03-03T06:26:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240cfd30 msgr2=0x7fd6240d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd621d9b700 1 --2- 172.21.15.3:0/1655788960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240cfd30 0x7fd6240d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 >> v1:172.21.15.32:6789/0 conn(0x7fd6241608d0 legacy=0x7fd6240cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6241600d0 con 0x7fd624153670 2023-03-03T06:26:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7fd614001670 con 0x7fd624153670 2023-03-03T06:26:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 >> v1:172.21.15.3:6789/0 conn(0x7fd624153670 legacy=0x7fd624071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd621d9b700 1 --2- 172.21.15.3:0/1655788960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c003a50 0x7fd60c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd621d9b700 1 --2- 172.21.15.3:0/1655788960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240cfd30 0x7fd60c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.070+0000 7fd621d9b700 1 --2- 172.21.15.3:0/1655788960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd60c004db0 0x7fd60c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240cd6c0 con 0x7fd60c004db0 2023-03-03T06:26:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c00ba30 con 0x7fd60c003a50 2023-03-03T06:26:50.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c00bbc0 con 0x7fd6240cfd30 2023-03-03T06:26:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd62359e700 1 --2- 172.21.15.3:0/1655788960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd60c004db0 0x7fd60c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd622d9d700 1 --2- 172.21.15.3:0/1655788960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c003a50 0x7fd60c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd62259c700 1 --2- 172.21.15.3:0/1655788960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240cfd30 0x7fd60c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd62359e700 1 -- 172.21.15.3:0/1655788960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240cfd30 msgr2=0x7fd60c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:50.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd62359e700 1 --2- 172.21.15.3:0/1655788960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240cfd30 0x7fd60c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd62359e700 1 -- 172.21.15.3:0/1655788960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c003a50 msgr2=0x7fd60c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:50.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd62359e700 1 --2- 172.21.15.3:0/1655788960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c003a50 0x7fd60c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd62359e700 1 -- 172.21.15.3:0/1655788960 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c0035d0 con 0x7fd60c004db0 2023-03-03T06:26:50.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd62259c700 1 --2- 172.21.15.3:0/1655788960 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240cfd30 0x7fd60c004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:50.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd62359e700 1 --2- 172.21.15.3:0/1655788960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd60c004db0 0x7fd60c005220 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd6240b8d60 tx=0x7fd60c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510cddb90331acc server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:50.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c00ebd0 con 0x7fd60c004db0 2023-03-03T06:26:50.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c00f040 con 0x7fd60c004db0 2023-03-03T06:26:50.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd621d9b700 1 -- 172.21.15.3:0/1655788960 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c014480 con 0x7fd60c004db0 2023-03-03T06:26:50.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd6288a6700 1 -- 172.21.15.3:0/1655788960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd60c004db0 msgr2=0x7fd60c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd6288a6700 1 --2- 172.21.15.3:0/1655788960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd60c004db0 0x7fd60c005220 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd6240b8d60 tx=0x7fd60c0058a0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd62359e700 1 -- 172.21.15.3:0/1655788960 reap_dead start 2023-03-03T06:26:50.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd6288a6700 1 -- 172.21.15.3:0/1655788960 shutdown_connections 2023-03-03T06:26:50.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd6288a6700 1 -- 172.21.15.3:0/1655788960 >> 172.21.15.3:0/1655788960 conn(0x7fd62406e5c0 msgr2=0x7fd62406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:50.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd6288a6700 1 -- 172.21.15.3:0/1655788960 shutdown_connections 2023-03-03T06:26:50.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.074+0000 7fd6288a6700 1 -- 172.21.15.3:0/1655788960 wait complete. 2023-03-03T06:26:50.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.078+0000 7fd6288a6700 1 Processor -- start 2023-03-03T06:26:50.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd6288a6700 1 -- start start 2023-03-03T06:26:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd6288a6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c004db0 0x7fd6240cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:50.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd6288a6700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240c5c90 0x7fd62415e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd6288a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6240c6100 0x7fd62415e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd6288a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6241554f0 con 0x7fd6240c6100 2023-03-03T06:26:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd6288a6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6241565b0 con 0x7fd60c004db0 2023-03-03T06:26:50.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd6288a6700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240cbef0 con 0x7fd6240c5c90 2023-03-03T06:26:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd62359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6240c6100 0x7fd62415e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd62359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6240c6100 0x7fd62415e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57582/0 (socket says 172.21.15.3:57582) 2023-03-03T06:26:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd62359e700 1 -- 172.21.15.3:0/587764461 learned_addr learned my addr 172.21.15.3:0/587764461 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd622d9d700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c004db0 0x7fd6240cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd62259c700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240c5c90 0x7fd62415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:50.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd62359e700 1 -- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240c5c90 msgr2=0x7fd62415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd62359e700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240c5c90 0x7fd62415e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd62359e700 1 -- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c004db0 msgr2=0x7fd6240cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd62359e700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c004db0 0x7fd6240cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.090 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd62359e700 1 -- 172.21.15.3:0/587764461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6240cdb30 con 0x7fd6240c6100 2023-03-03T06:26:50.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.082+0000 7fd62359e700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6240c6100 0x7fd62415e650 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd60c005820 tx=0x7fd61c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:50.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.086+0000 7fd60b7fe700 1 -- 172.21.15.3:0/587764461 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c007000 con 0x7fd6240c6100 2023-03-03T06:26:50.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.086+0000 7fd60b7fe700 1 -- 172.21.15.3:0/587764461 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c014d60 con 0x7fd6240c6100 2023-03-03T06:26:50.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.086+0000 7fd60b7fe700 1 -- 172.21.15.3:0/587764461 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c003c80 con 0x7fd6240c6100 2023-03-03T06:26:50.091 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.086+0000 7fd6288a6700 1 -- 172.21.15.3:0/587764461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6240cd9a0 con 0x7fd6240c6100 2023-03-03T06:26:50.092 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.086+0000 7fd6288a6700 1 -- 172.21.15.3:0/587764461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6240cf200 con 0x7fd6240c6100 2023-03-03T06:26:50.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.086+0000 7fd6288a6700 1 -- 172.21.15.3:0/587764461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd624073e90 con 0x7fd6240c6100 2023-03-03T06:26:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.090+0000 7fd60b7fe700 1 -- 172.21.15.3:0/587764461 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd61c004030 con 0x7fd6240c6100 2023-03-03T06:26:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.090+0000 7fd60b7fe700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd610064ee0 0x7fd610067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.090+0000 7fd60b7fe700 1 -- 172.21.15.3:0/587764461 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd61c083a90 con 0x7fd6240c6100 2023-03-03T06:26:50.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.090+0000 7fd622d9d700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd610064ee0 0x7fd610067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:50.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.094+0000 7fd622d9d700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd610064ee0 0x7fd610067390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd614005ae0 tx=0x7fd614009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:50.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.098+0000 7fd60b7fe700 1 -- 172.21.15.3:0/587764461 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd61c051d70 con 0x7fd6240c6100 2023-03-03T06:26:50.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.330+0000 7fd6288a6700 1 -- 172.21.15.3:0/587764461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fd624071130 con 0x7fd6240c6100 2023-03-03T06:26:50.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.338+0000 7fd60b7fe700 1 -- 172.21.15.3:0/587764461 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd61c0180a0 con 0x7fd6240c6100 2023-03-03T06:26:50.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.342+0000 7fd6288a6700 1 -- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd610064ee0 msgr2=0x7fd610067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.342+0000 7fd6288a6700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd610064ee0 0x7fd610067390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd614005ae0 tx=0x7fd614009040 comp rx=0 tx=0).stop 2023-03-03T06:26:50.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.342+0000 7fd6288a6700 1 -- 172.21.15.3:0/587764461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6240c6100 msgr2=0x7fd62415e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.342+0000 7fd6288a6700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6240c6100 0x7fd62415e650 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd60c005820 tx=0x7fd61c009750 comp rx=0 tx=0).stop 2023-03-03T06:26:50.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.342+0000 7fd6288a6700 1 -- 172.21.15.3:0/587764461 shutdown_connections 2023-03-03T06:26:50.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.342+0000 7fd6288a6700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd610064ee0 0x7fd610067390 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.342+0000 7fd6288a6700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6240c6100 0x7fd62415e650 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.350 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.342+0000 7fd6288a6700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd6240c5c90 0x7fd62415e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.342+0000 7fd6288a6700 1 --2- 172.21.15.3:0/587764461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd60c004db0 0x7fd6240cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.342+0000 7fd6288a6700 1 -- 172.21.15.3:0/587764461 >> 172.21.15.3:0/587764461 conn(0x7fd62406e5c0 msgr2=0x7fd624152490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:50.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.346+0000 7fd6288a6700 1 -- 172.21.15.3:0/587764461 shutdown_connections 2023-03-03T06:26:50.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.346+0000 7fd6288a6700 1 -- 172.21.15.3:0/587764461 wait complete. 2023-03-03T06:26:50.382 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-03-03T06:26:50.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.582+0000 7f9d36620700 1 Processor -- start 2023-03-03T06:26:50.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.582+0000 7f9d36620700 1 -- start start 2023-03-03T06:26:50.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.586+0000 7f9d36620700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d300b0cd0 0x7f9d300c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:50.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.586+0000 7f9d36620700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d300b1120 con 0x7f9d300ca280 2023-03-03T06:26:50.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.586+0000 7f9d36620700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3015e190 con 0x7f9d300cb1b0 2023-03-03T06:26:50.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.586+0000 7f9d36620700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3015a7d0 con 0x7f9d300b0cd0 2023-03-03T06:26:50.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.586+0000 7f9d3561e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9d300cb1b0 0x7f9d300cb580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:57874/0 (socket says 172.21.15.3:57874) 2023-03-03T06:26:50.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.586+0000 7f9d3561e700 1 -- 172.21.15.3:0/3740694380 learned_addr learned my addr 172.21.15.3:0/3740694380 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:50.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.586+0000 7f9d34e1d700 1 --2- 172.21.15.3:0/3740694380 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d300b0cd0 0x7f9d300c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:50.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.590+0000 7f9d34e1d700 1 -- 172.21.15.3:0/3740694380 >> v1:172.21.15.32:6789/0 conn(0x7f9d300cb1b0 legacy=0x7f9d300cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.590+0000 7f9d34e1d700 1 -- 172.21.15.3:0/3740694380 >> v1:172.21.15.3:6789/0 conn(0x7f9d300ca280 legacy=0x7f9d300ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.590+0000 7f9d34e1d700 1 -- 172.21.15.3:0/3740694380 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3015e470 con 0x7f9d300b0cd0 2023-03-03T06:26:50.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.590+0000 7f9d34e1d700 1 --2- 172.21.15.3:0/3740694380 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d300b0cd0 0x7f9d300c9d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009880 tx=0x7f9d20009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8759d9681a761ff server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:50.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.590+0000 7f9d27fff700 1 -- 172.21.15.3:0/3740694380 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20011830 con 0x7f9d300b0cd0 2023-03-03T06:26:50.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.590+0000 7f9d27fff700 1 -- 172.21.15.3:0/3740694380 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d20011990 con 0x7f9d300b0cd0 2023-03-03T06:26:50.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.590+0000 7f9d27fff700 1 -- 172.21.15.3:0/3740694380 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20011d20 con 0x7f9d300b0cd0 2023-03-03T06:26:50.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.590+0000 7f9d36620700 1 -- 172.21.15.3:0/3740694380 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d300b0cd0 msgr2=0x7f9d300c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.590+0000 7f9d36620700 1 --2- 172.21.15.3:0/3740694380 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d300b0cd0 0x7f9d300c9d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009880 tx=0x7f9d20009b90 comp rx=0 tx=0).stop 2023-03-03T06:26:50.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.590+0000 7f9d36620700 1 -- 172.21.15.3:0/3740694380 shutdown_connections 2023-03-03T06:26:50.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.590+0000 7f9d36620700 1 --2- 172.21.15.3:0/3740694380 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d300b0cd0 0x7f9d300c9d40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.590+0000 7f9d36620700 1 -- 172.21.15.3:0/3740694380 >> 172.21.15.3:0/3740694380 conn(0x7f9d30070a30 msgr2=0x7f9d30070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:50.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.594+0000 7f9d36620700 1 -- 172.21.15.3:0/3740694380 shutdown_connections 2023-03-03T06:26:50.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.594+0000 7f9d36620700 1 -- 172.21.15.3:0/3740694380 wait complete. 2023-03-03T06:26:50.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.594+0000 7f9d36620700 1 Processor -- start 2023-03-03T06:26:50.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.594+0000 7f9d36620700 1 -- start start 2023-03-03T06:26:50.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d36620700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d300b0cd0 0x7f9d30150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:50.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d36620700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d300ca280 0x7f9d30150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:50.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d36620700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d300cb1b0 0x7f9d3015b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:50.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d36620700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30150670 con 0x7f9d300cb1b0 2023-03-03T06:26:50.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d36620700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300ce220 con 0x7f9d300b0cd0 2023-03-03T06:26:50.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d36620700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3015c600 con 0x7f9d300ca280 2023-03-03T06:26:50.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d35e1f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d300cb1b0 0x7f9d3015b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:50.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d35e1f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d300cb1b0 0x7f9d3015b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52172/0 (socket says 172.21.15.3:52172) 2023-03-03T06:26:50.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d35e1f700 1 -- 172.21.15.3:0/3816327308 learned_addr learned my addr 172.21.15.3:0/3816327308 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:50.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d34e1d700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d300ca280 0x7f9d30150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:50.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d3561e700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d300b0cd0 0x7f9d30150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:50.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d35e1f700 1 -- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d300ca280 msgr2=0x7f9d30150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d35e1f700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d300ca280 0x7f9d30150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d35e1f700 1 -- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d300b0cd0 msgr2=0x7f9d30150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d35e1f700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d300b0cd0 0x7f9d30150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d35e1f700 1 -- 172.21.15.3:0/3816327308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d200094b0 con 0x7f9d300cb1b0 2023-03-03T06:26:50.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d3561e700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d300b0cd0 0x7f9d30150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:50.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d35e1f700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d300cb1b0 0x7f9d3015b780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800cb50 tx=0x7f9d2800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:50.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d25ffb700 1 -- 172.21.15.3:0/3816327308 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28003c20 con 0x7f9d300cb1b0 2023-03-03T06:26:50.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d25ffb700 1 -- 172.21.15.3:0/3816327308 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d28003d80 con 0x7f9d300cb1b0 2023-03-03T06:26:50.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.598+0000 7f9d36620700 1 -- 172.21.15.3:0/3816327308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3015ea30 con 0x7f9d300cb1b0 2023-03-03T06:26:50.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.602+0000 7f9d25ffb700 1 -- 172.21.15.3:0/3816327308 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2800ab70 con 0x7f9d300cb1b0 2023-03-03T06:26:50.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.602+0000 7f9d36620700 1 -- 172.21.15.3:0/3816327308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3015e750 con 0x7f9d300cb1b0 2023-03-03T06:26:50.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.602+0000 7f9d36620700 1 -- 172.21.15.3:0/3816327308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d30073e90 con 0x7f9d300cb1b0 2023-03-03T06:26:50.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.606+0000 7f9d25ffb700 1 -- 172.21.15.3:0/3816327308 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9d28010050 con 0x7f9d300cb1b0 2023-03-03T06:26:50.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.610+0000 7f9d25ffb700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9d14064ee0 0x7f9d14067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:50.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.610+0000 7f9d25ffb700 1 -- 172.21.15.3:0/3816327308 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f9d28081cc0 con 0x7f9d300cb1b0 2023-03-03T06:26:50.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.610+0000 7f9d3561e700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9d14064ee0 0x7f9d14067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:50.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.610+0000 7f9d3561e700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9d14064ee0 0x7f9d14067390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c005930 tx=0x7f9d2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:50.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.614+0000 7f9d25ffb700 1 -- 172.21.15.3:0/3816327308 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9d28088080 con 0x7f9d300cb1b0 2023-03-03T06:26:50.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.858+0000 7f9d36620700 1 -- 172.21.15.3:0/3816327308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f9d300c93b0 con 0x7f9d300cb1b0 2023-03-03T06:26:50.872 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.866+0000 7f9d25ffb700 1 -- 172.21.15.3:0/3816327308 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9d2804ff80 con 0x7f9d300cb1b0 2023-03-03T06:26:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.870+0000 7f9d36620700 1 -- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9d14064ee0 msgr2=0x7f9d14067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.870+0000 7f9d36620700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9d14064ee0 0x7f9d14067390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c005930 tx=0x7f9d2c009040 comp rx=0 tx=0).stop 2023-03-03T06:26:50.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.870+0000 7f9d36620700 1 -- 172.21.15.3:0/3816327308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d300cb1b0 msgr2=0x7f9d3015b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:50.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.870+0000 7f9d36620700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d300cb1b0 0x7f9d3015b780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800cb50 tx=0x7f9d2800cf10 comp rx=0 tx=0).stop 2023-03-03T06:26:50.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.870+0000 7f9d36620700 1 -- 172.21.15.3:0/3816327308 shutdown_connections 2023-03-03T06:26:50.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.870+0000 7f9d36620700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9d14064ee0 0x7f9d14067390 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.870+0000 7f9d36620700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d300cb1b0 0x7f9d3015b780 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.870+0000 7f9d36620700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f9d300ca280 0x7f9d30150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.870+0000 7f9d36620700 1 --2- 172.21.15.3:0/3816327308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d300b0cd0 0x7f9d30150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:50.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.870+0000 7f9d36620700 1 -- 172.21.15.3:0/3816327308 >> 172.21.15.3:0/3816327308 conn(0x7f9d30070a30 msgr2=0x7f9d3006d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:50.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.870+0000 7f9d36620700 1 -- 172.21.15.3:0/3816327308 shutdown_connections 2023-03-03T06:26:50.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:50.870+0000 7f9d36620700 1 -- 172.21.15.3:0/3816327308 wait complete. 2023-03-03T06:26:50.910 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-03-03T06:26:51.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.110+0000 7fa8d71ce700 1 Processor -- start 2023-03-03T06:26:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.110+0000 7fa8d71ce700 1 -- start start 2023-03-03T06:26:51.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.114+0000 7fa8d71ce700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8d00adf20 0x7fa8d00b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:51.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.118+0000 7fa8d71ce700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d00c9a20 con 0x7fa8d015dec0 2023-03-03T06:26:51.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.118+0000 7fa8d71ce700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d00c9fe0 con 0x7fa8d00cb890 2023-03-03T06:26:51.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.118+0000 7fa8d71ce700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d00ae2f0 con 0x7fa8d00adf20 2023-03-03T06:26:51.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.118+0000 7fa8d61cc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa8d00adf20 0x7fa8d00b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:51.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.118+0000 7fa8d59cb700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa8d015dec0 0x7fa8d00cb180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47300/0 (socket says 172.21.15.3:47300) 2023-03-03T06:26:51.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.118+0000 7fa8d59cb700 1 -- 172.21.15.3:0/2588714899 learned_addr learned my addr 172.21.15.3:0/2588714899 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:51.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.118+0000 7fa8d61cc700 1 -- 172.21.15.3:0/2588714899 >> v1:172.21.15.32:6789/0 conn(0x7fa8d00cb890 legacy=0x7fa8d00ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.118+0000 7fa8d61cc700 1 -- 172.21.15.3:0/2588714899 >> v1:172.21.15.3:6789/0 conn(0x7fa8d015dec0 legacy=0x7fa8d00cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.118+0000 7fa8d61cc700 1 -- 172.21.15.3:0/2588714899 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d015a7d0 con 0x7fa8d00adf20 2023-03-03T06:26:51.124 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.118+0000 7fa8d61cc700 1 --2- 172.21.15.3:0/2588714899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d00adf20 0x7fa8d00b1630 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009850 tx=0x7fa8cc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb476cf24f7ec2c server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.118+0000 7fa8d51ca700 1 -- 172.21.15.3:0/2588714899 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc004230 con 0x7fa8d00adf20 2023-03-03T06:26:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.118+0000 7fa8d51ca700 1 -- 172.21.15.3:0/2588714899 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc004390 con 0x7fa8d00adf20 2023-03-03T06:26:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.118+0000 7fa8d51ca700 1 -- 172.21.15.3:0/2588714899 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc0036c0 con 0x7fa8d00adf20 2023-03-03T06:26:51.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.122+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2588714899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d00adf20 msgr2=0x7fa8d00b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:51.126 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.122+0000 7fa8d71ce700 1 --2- 172.21.15.3:0/2588714899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d00adf20 0x7fa8d00b1630 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009850 tx=0x7fa8cc009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.122+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2588714899 shutdown_connections 2023-03-03T06:26:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.122+0000 7fa8d71ce700 1 --2- 172.21.15.3:0/2588714899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d00adf20 0x7fa8d00b1630 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.122+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2588714899 >> 172.21.15.3:0/2588714899 conn(0x7fa8d0070a30 msgr2=0x7fa8d0070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.122+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2588714899 shutdown_connections 2023-03-03T06:26:51.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.122+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2588714899 wait complete. 2023-03-03T06:26:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.126+0000 7fa8d71ce700 1 Processor -- start 2023-03-03T06:26:51.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.126+0000 7fa8d71ce700 1 -- start start 2023-03-03T06:26:51.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.126+0000 7fa8d71ce700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d00adf20 0x7fa8d00b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:51.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.126+0000 7fa8d71ce700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8d00cb890 0x7fa8d0158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:51.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.126+0000 7fa8d71ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8d015dec0 0x7fa8d00b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:51.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.126+0000 7fa8d71ce700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00b1b70 con 0x7fa8d00adf20 2023-03-03T06:26:51.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.126+0000 7fa8d71ce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00c3250 con 0x7fa8d015dec0 2023-03-03T06:26:51.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.126+0000 7fa8d71ce700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00c3530 con 0x7fa8d00cb890 2023-03-03T06:26:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.126+0000 7fa8d61cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d00adf20 0x7fa8d00b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.126+0000 7fa8d61cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d00adf20 0x7fa8d00b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52186/0 (socket says 172.21.15.3:52186) 2023-03-03T06:26:51.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.126+0000 7fa8d61cc700 1 -- 172.21.15.3:0/2283122867 learned_addr learned my addr 172.21.15.3:0/2283122867 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8d59cb700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8d00cb890 0x7fa8d0158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8d69cd700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8d015dec0 0x7fa8d00b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8d61cc700 1 -- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8d00cb890 msgr2=0x7fa8d0158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8d61cc700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8d00cb890 0x7fa8d0158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8d61cc700 1 -- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8d015dec0 msgr2=0x7fa8d00b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8d61cc700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8d015dec0 0x7fa8d00b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8d61cc700 1 -- 172.21.15.3:0/2283122867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc0094b0 con 0x7fa8d00adf20 2023-03-03T06:26:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8d61cc700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d00adf20 0x7fa8d00b1630 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc000c00 tx=0x7fa8cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8c6ffd700 1 -- 172.21.15.3:0/2283122867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc006ea0 con 0x7fa8d00adf20 2023-03-03T06:26:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8c6ffd700 1 -- 172.21.15.3:0/2283122867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc01f070 con 0x7fa8d00adf20 2023-03-03T06:26:51.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8c6ffd700 1 -- 172.21.15.3:0/2283122867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc003ea0 con 0x7fa8d00adf20 2023-03-03T06:26:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2283122867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d015ebf0 con 0x7fa8d00adf20 2023-03-03T06:26:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2283122867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d015e910 con 0x7fa8d00adf20 2023-03-03T06:26:51.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.130+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2283122867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d00c9c70 con 0x7fa8d00adf20 2023-03-03T06:26:51.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.134+0000 7fa8c6ffd700 1 -- 172.21.15.3:0/2283122867 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa8cc02a020 con 0x7fa8d00adf20 2023-03-03T06:26:51.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.138+0000 7fa8c6ffd700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa8b4064ee0 0x7fa8b4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.138+0000 7fa8c6ffd700 1 -- 172.21.15.3:0/2283122867 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fa8cc083520 con 0x7fa8d00adf20 2023-03-03T06:26:51.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.138+0000 7fa8d59cb700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa8b4064ee0 0x7fa8b4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:51.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.138+0000 7fa8d59cb700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa8b4064ee0 0x7fa8b4067390 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa8d01f5580 tx=0x7fa8c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:51.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.142+0000 7fa8c6ffd700 1 -- 172.21.15.3:0/2283122867 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa8cc0517e0 con 0x7fa8d00adf20 2023-03-03T06:26:51.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.382+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2283122867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fa8d00c9c70 con 0x7fa8d00adf20 2023-03-03T06:26:51.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.394+0000 7fa8c6ffd700 1 -- 172.21.15.3:0/2283122867 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa8cc054e00 con 0x7fa8d00adf20 2023-03-03T06:26:51.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.398+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa8b4064ee0 msgr2=0x7fa8b4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.398+0000 7fa8d71ce700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa8b4064ee0 0x7fa8b4067390 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa8d01f5580 tx=0x7fa8c0009040 comp rx=0 tx=0).stop 2023-03-03T06:26:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.398+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2283122867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d00adf20 msgr2=0x7fa8d00b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.398+0000 7fa8d71ce700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d00adf20 0x7fa8d00b1630 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc000c00 tx=0x7fa8cc004770 comp rx=0 tx=0).stop 2023-03-03T06:26:51.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.398+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2283122867 shutdown_connections 2023-03-03T06:26:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.398+0000 7fa8d71ce700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa8b4064ee0 0x7fa8b4067390 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.398+0000 7fa8d71ce700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8d015dec0 0x7fa8d00b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.398+0000 7fa8d71ce700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa8d00cb890 0x7fa8d0158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.398+0000 7fa8d71ce700 1 --2- 172.21.15.3:0/2283122867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8d00adf20 0x7fa8d00b1630 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.398+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2283122867 >> 172.21.15.3:0/2283122867 conn(0x7fa8d0070a30 msgr2=0x7fa8d006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:51.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.398+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2283122867 shutdown_connections 2023-03-03T06:26:51.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.398+0000 7fa8d71ce700 1 -- 172.21.15.3:0/2283122867 wait complete. 2023-03-03T06:26:51.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-03-03T06:26:51.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.642+0000 7fd3f60dc700 1 Processor -- start 2023-03-03T06:26:51.649 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.642+0000 7fd3f60dc700 1 -- start start 2023-03-03T06:26:51.650 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.646+0000 7fd3f60dc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3f00adec0 0x7fd3f016b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.646+0000 7fd3f60dc700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f00ce190 con 0x7fd3f00aea70 2023-03-03T06:26:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.646+0000 7fd3f60dc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f00c99d0 con 0x7fd3f0160880 2023-03-03T06:26:51.651 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.646+0000 7fd3f60dc700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f00cd580 con 0x7fd3f00adec0 2023-03-03T06:26:51.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.646+0000 7fd3f48d9700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd3f00aea70 0x7fd3f00ad7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47316/0 (socket says 172.21.15.3:47316) 2023-03-03T06:26:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.646+0000 7fd3f58db700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd3f00adec0 0x7fd3f016b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.646+0000 7fd3f48d9700 1 -- 172.21.15.3:0/1734228731 learned_addr learned my addr 172.21.15.3:0/1734228731 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.646+0000 7fd3f58db700 1 -- 172.21.15.3:0/1734228731 >> v1:172.21.15.32:6789/0 conn(0x7fd3f0160880 legacy=0x7fd3f00c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:51.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.646+0000 7fd3f58db700 1 -- 172.21.15.3:0/1734228731 >> v1:172.21.15.3:6789/0 conn(0x7fd3f00aea70 legacy=0x7fd3f00ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:51.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.646+0000 7fd3f58db700 1 -- 172.21.15.3:0/1734228731 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f00cd860 con 0x7fd3f00adec0 2023-03-03T06:26:51.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.646+0000 7fd3f58db700 1 --2- 172.21.15.3:0/1734228731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3f00adec0 0x7fd3f016b820 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009d90 tx=0x7fd3e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef736e83c8ef45e9 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:51.655 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.646+0000 7fd3e7fff700 1 -- 172.21.15.3:0/1734228731 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800e070 con 0x7fd3f00adec0 2023-03-03T06:26:51.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.650+0000 7fd3e7fff700 1 -- 172.21.15.3:0/1734228731 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e800bbd0 con 0x7fd3f00adec0 2023-03-03T06:26:51.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.650+0000 7fd3e7fff700 1 -- 172.21.15.3:0/1734228731 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8012430 con 0x7fd3f00adec0 2023-03-03T06:26:51.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.650+0000 7fd3f60dc700 1 -- 172.21.15.3:0/1734228731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3f00adec0 msgr2=0x7fd3f016b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:51.656 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.650+0000 7fd3f60dc700 1 --2- 172.21.15.3:0/1734228731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3f00adec0 0x7fd3f016b820 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009d90 tx=0x7fd3e800b330 comp rx=0 tx=0).stop 2023-03-03T06:26:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.650+0000 7fd3f60dc700 1 -- 172.21.15.3:0/1734228731 shutdown_connections 2023-03-03T06:26:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.650+0000 7fd3f60dc700 1 --2- 172.21.15.3:0/1734228731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3f00adec0 0x7fd3f016b820 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.657 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.650+0000 7fd3f60dc700 1 -- 172.21.15.3:0/1734228731 >> 172.21.15.3:0/1734228731 conn(0x7fd3f0070a30 msgr2=0x7fd3f0070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.650+0000 7fd3f60dc700 1 -- 172.21.15.3:0/1734228731 shutdown_connections 2023-03-03T06:26:51.658 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.650+0000 7fd3f60dc700 1 -- 172.21.15.3:0/1734228731 wait complete. 2023-03-03T06:26:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f60dc700 1 Processor -- start 2023-03-03T06:26:51.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f60dc700 1 -- start start 2023-03-03T06:26:51.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f60dc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3f00adec0 0x7fd3f01f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:51.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f60dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3f00aea70 0x7fd3f0157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:51.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f60dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f0160880 0x7fd3f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:51.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f60dc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f0158a10 con 0x7fd3f00aea70 2023-03-03T06:26:51.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f60dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00b23e0 con 0x7fd3f0160880 2023-03-03T06:26:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f60dc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00b17d0 con 0x7fd3f00adec0 2023-03-03T06:26:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f48d9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3f00aea70 0x7fd3f0157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f48d9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3f00aea70 0x7fd3f0157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52202/0 (socket says 172.21.15.3:52202) 2023-03-03T06:26:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f48d9700 1 -- 172.21.15.3:0/3250230837 learned_addr learned my addr 172.21.15.3:0/3250230837 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f50da700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3f00adec0 0x7fd3f01f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f48d9700 1 -- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3f00adec0 msgr2=0x7fd3f01f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f58db700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f0160880 0x7fd3f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f48d9700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3f00adec0 0x7fd3f01f5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f48d9700 1 -- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f0160880 msgr2=0x7fd3f006d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f48d9700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f0160880 0x7fd3f006d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f48d9700 1 -- 172.21.15.3:0/3250230837 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e8009a70 con 0x7fd3f00aea70 2023-03-03T06:26:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f50da700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3f00adec0 0x7fd3f01f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:51.666 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.654+0000 7fd3f48d9700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3f00aea70 0x7fd3f0157990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000c500 tx=0x7fd3e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:51.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.658+0000 7fd3e5ffb700 1 -- 172.21.15.3:0/3250230837 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0005d90 con 0x7fd3f00aea70 2023-03-03T06:26:51.667 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.658+0000 7fd3e5ffb700 1 -- 172.21.15.3:0/3250230837 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e00070d0 con 0x7fd3f00aea70 2023-03-03T06:26:51.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.658+0000 7fd3e5ffb700 1 -- 172.21.15.3:0/3250230837 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0007440 con 0x7fd3f00aea70 2023-03-03T06:26:51.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.658+0000 7fd3f60dc700 1 -- 172.21.15.3:0/3250230837 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f00cde20 con 0x7fd3f00aea70 2023-03-03T06:26:51.668 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.658+0000 7fd3f60dc700 1 -- 172.21.15.3:0/3250230837 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f00cdb40 con 0x7fd3f00aea70 2023-03-03T06:26:51.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.658+0000 7fd3f60dc700 1 -- 172.21.15.3:0/3250230837 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f00b2550 con 0x7fd3f00aea70 2023-03-03T06:26:51.669 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.662+0000 7fd3e5ffb700 1 -- 172.21.15.3:0/3250230837 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd3e0019030 con 0x7fd3f00aea70 2023-03-03T06:26:51.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.662+0000 7fd3e5ffb700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd3d4064f30 0x7fd3d40673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:51.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.662+0000 7fd3e5ffb700 1 -- 172.21.15.3:0/3250230837 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd3e0080f60 con 0x7fd3f00aea70 2023-03-03T06:26:51.670 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.662+0000 7fd3f50da700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd3d4064f30 0x7fd3d40673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:51.671 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.666+0000 7fd3f50da700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd3d4064f30 0x7fd3d40673e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec000f60 tx=0x7fd3ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:51.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.670+0000 7fd3e5ffb700 1 -- 172.21.15.3:0/3250230837 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd3e004a400 con 0x7fd3f00aea70 2023-03-03T06:26:51.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.910+0000 7fd3f60dc700 1 -- 172.21.15.3:0/3250230837 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fd3f0073e90 con 0x7fd3f00aea70 2023-03-03T06:26:51.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.922+0000 7fd3e5ffb700 1 -- 172.21.15.3:0/3250230837 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd3e004f350 con 0x7fd3f00aea70 2023-03-03T06:26:51.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.926+0000 7fd3f60dc700 1 -- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd3d4064f30 msgr2=0x7fd3d40673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:51.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.926+0000 7fd3f60dc700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd3d4064f30 0x7fd3d40673e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec000f60 tx=0x7fd3ec009040 comp rx=0 tx=0).stop 2023-03-03T06:26:51.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.926+0000 7fd3f60dc700 1 -- 172.21.15.3:0/3250230837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3f00aea70 msgr2=0x7fd3f0157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:51.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.926+0000 7fd3f60dc700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3f00aea70 0x7fd3f0157990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000c500 tx=0x7fd3e000c8c0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.926+0000 7fd3f60dc700 1 -- 172.21.15.3:0/3250230837 shutdown_connections 2023-03-03T06:26:51.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.926+0000 7fd3f60dc700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd3d4064f30 0x7fd3d40673e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.926+0000 7fd3f60dc700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f0160880 0x7fd3f006d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.926+0000 7fd3f60dc700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3f00aea70 0x7fd3f0157990 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.926+0000 7fd3f60dc700 1 --2- 172.21.15.3:0/3250230837 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd3f00adec0 0x7fd3f01f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:51.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.926+0000 7fd3f60dc700 1 -- 172.21.15.3:0/3250230837 >> 172.21.15.3:0/3250230837 conn(0x7fd3f0070a30 msgr2=0x7fd3f00b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:51.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.926+0000 7fd3f60dc700 1 -- 172.21.15.3:0/3250230837 shutdown_connections 2023-03-03T06:26:51.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:51.926+0000 7fd3f60dc700 1 -- 172.21.15.3:0/3250230837 wait complete. 2023-03-03T06:26:51.965 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-03-03T06:26:52.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.166+0000 7fc35759e700 1 Processor -- start 2023-03-03T06:26:52.171 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.166+0000 7fc35759e700 1 -- start start 2023-03-03T06:26:52.175 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc35759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3581608d0 0x7fc3580cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:52.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc35759e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc3580cfb60 0x7fc3580d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:52.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc35759e700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc358160440 con 0x7fc358153670 2023-03-03T06:26:52.176 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc35759e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3580cffa0 con 0x7fc3580cfb60 2023-03-03T06:26:52.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc35759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3580cd3e0 con 0x7fc3581608d0 2023-03-03T06:26:52.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc35659c700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc358153670 0x7fc358071350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47328/0 (socket says 172.21.15.3:47328) 2023-03-03T06:26:52.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc35659c700 1 -- 172.21.15.3:0/1215010316 learned_addr learned my addr 172.21.15.3:0/1215010316 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:52.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc356d9d700 1 --2- 172.21.15.3:0/1215010316 >> v2:172.21.15.3:3300/0 conn(0x7fc3580cfb60 0x7fc3580d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:52.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc355d9b700 1 --2- 172.21.15.3:0/1215010316 >> v2:172.21.15.32:3300/0 conn(0x7fc3581608d0 0x7fc3580cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:52.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc35559a700 1 -- 172.21.15.3:0/1215010316 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606348619 0 0) 0x7fc358160440 con 0x7fc358153670 2023-03-03T06:26:52.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc35559a700 1 -- 172.21.15.3:0/1215010316 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3400035d0 con 0x7fc358153670 2023-03-03T06:26:52.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc356d9d700 1 -- 172.21.15.3:0/1215010316 >> v2:172.21.15.32:3300/0 conn(0x7fc3581608d0 msgr2=0x7fc3580cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:52.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc356d9d700 1 --2- 172.21.15.3:0/1215010316 >> v2:172.21.15.32:3300/0 conn(0x7fc3581608d0 0x7fc3580cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.178 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc356d9d700 1 -- 172.21.15.3:0/1215010316 >> v1:172.21.15.3:6789/0 conn(0x7fc358153670 legacy=0x7fc358071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.170+0000 7fc356d9d700 1 -- 172.21.15.3:0/1215010316 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3580cd6c0 con 0x7fc3580cfb60 2023-03-03T06:26:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.174+0000 7fc356d9d700 1 --2- 172.21.15.3:0/1215010316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3580cfb60 0x7fc3580d0b10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc350009d90 tx=0x7fc35000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779175a87f42ff19 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.174+0000 7fc35559a700 1 -- 172.21.15.3:0/1215010316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35000e070 con 0x7fc3580cfb60 2023-03-03T06:26:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.174+0000 7fc35559a700 1 -- 172.21.15.3:0/1215010316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35000bae0 con 0x7fc3580cfb60 2023-03-03T06:26:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.174+0000 7fc35559a700 1 -- 172.21.15.3:0/1215010316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35000be90 con 0x7fc3580cfb60 2023-03-03T06:26:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.174+0000 7fc35759e700 1 -- 172.21.15.3:0/1215010316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3580cfb60 msgr2=0x7fc3580d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.174+0000 7fc35759e700 1 --2- 172.21.15.3:0/1215010316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3580cfb60 0x7fc3580d0b10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc350009d90 tx=0x7fc35000b330 comp rx=0 tx=0).stop 2023-03-03T06:26:52.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.174+0000 7fc35759e700 1 -- 172.21.15.3:0/1215010316 shutdown_connections 2023-03-03T06:26:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.174+0000 7fc35759e700 1 --2- 172.21.15.3:0/1215010316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3580cfb60 0x7fc3580d0b10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.174+0000 7fc35759e700 1 --2- 172.21.15.3:0/1215010316 >> v2:172.21.15.32:3300/0 conn(0x7fc3581608d0 0x7fc3580cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.174+0000 7fc35759e700 1 -- 172.21.15.3:0/1215010316 >> 172.21.15.3:0/1215010316 conn(0x7fc35806e5c0 msgr2=0x7fc35806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.174+0000 7fc35759e700 1 -- 172.21.15.3:0/1215010316 shutdown_connections 2023-03-03T06:26:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.174+0000 7fc35759e700 1 -- 172.21.15.3:0/1215010316 wait complete. 2023-03-03T06:26:52.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.178+0000 7fc35759e700 1 Processor -- start 2023-03-03T06:26:52.183 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.178+0000 7fc35759e700 1 -- start start 2023-03-03T06:26:52.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.178+0000 7fc35759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3580cfb60 0x7fc358152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:52.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.178+0000 7fc35759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc358153670 0x7fc35814f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:52.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.178+0000 7fc35759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3581608d0 0x7fc3581f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:52.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.178+0000 7fc35759e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35815b920 con 0x7fc358153670 2023-03-03T06:26:52.185 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.178+0000 7fc35759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc358152680 con 0x7fc3580cfb60 2023-03-03T06:26:52.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.178+0000 7fc35759e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3581f2b00 con 0x7fc3581608d0 2023-03-03T06:26:52.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.178+0000 7fc355d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc358153670 0x7fc35814f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:52.186 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc355d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc358153670 0x7fc35814f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52216/0 (socket says 172.21.15.3:52216) 2023-03-03T06:26:52.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc355d9b700 1 -- 172.21.15.3:0/52942282 learned_addr learned my addr 172.21.15.3:0/52942282 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:52.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc35659c700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3580cfb60 0x7fc358152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:52.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc356d9d700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3581608d0 0x7fc3581f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:52.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc355d9b700 1 -- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3581608d0 msgr2=0x7fc3581f25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:52.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc355d9b700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3581608d0 0x7fc3581f25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc355d9b700 1 -- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3580cfb60 msgr2=0x7fc358152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:52.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc355d9b700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3580cfb60 0x7fc358152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc355d9b700 1 -- 172.21.15.3:0/52942282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc350009a70 con 0x7fc358153670 2023-03-03T06:26:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc355d9b700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc358153670 0x7fc35814f070 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00e870 tx=0x7fc34c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc346ffd700 1 -- 172.21.15.3:0/52942282 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c0095a0 con 0x7fc358153670 2023-03-03T06:26:52.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc346ffd700 1 -- 172.21.15.3:0/52942282 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c013680 con 0x7fc358153670 2023-03-03T06:26:52.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc346ffd700 1 -- 172.21.15.3:0/52942282 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c0139f0 con 0x7fc358153670 2023-03-03T06:26:52.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc35759e700 1 -- 172.21.15.3:0/52942282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3580cd9a0 con 0x7fc358153670 2023-03-03T06:26:52.189 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.182+0000 7fc35759e700 1 -- 172.21.15.3:0/52942282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3581600d0 con 0x7fc358153670 2023-03-03T06:26:52.190 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.186+0000 7fc35759e700 1 -- 172.21.15.3:0/52942282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3580cdb30 con 0x7fc358153670 2023-03-03T06:26:52.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.186+0000 7fc346ffd700 1 -- 172.21.15.3:0/52942282 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc34c019030 con 0x7fc358153670 2023-03-03T06:26:52.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.190+0000 7fc346ffd700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc338064ee0 0x7fc338067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:52.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.190+0000 7fc346ffd700 1 -- 172.21.15.3:0/52942282 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fc34c081ab0 con 0x7fc358153670 2023-03-03T06:26:52.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.190+0000 7fc35659c700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc338064ee0 0x7fc338067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:52.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.194+0000 7fc35659c700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc338064ee0 0x7fc338067390 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc348007fd0 tx=0x7fc348007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:52.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.198+0000 7fc346ffd700 1 -- 172.21.15.3:0/52942282 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc34c04fd70 con 0x7fc358153670 2023-03-03T06:26:52.443 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.438+0000 7fc35759e700 1 -- 172.21.15.3:0/52942282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fc358152830 con 0x7fc358153670 2023-03-03T06:26:52.453 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.446+0000 7fc346ffd700 1 -- 172.21.15.3:0/52942282 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc358152830 con 0x7fc358153670 2023-03-03T06:26:52.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.450+0000 7fc35759e700 1 -- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc338064ee0 msgr2=0x7fc338067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:52.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.450+0000 7fc35759e700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc338064ee0 0x7fc338067390 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc348007fd0 tx=0x7fc348007de0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.457 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.450+0000 7fc35759e700 1 -- 172.21.15.3:0/52942282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc358153670 msgr2=0x7fc35814f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.450+0000 7fc35759e700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc358153670 0x7fc35814f070 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00e870 tx=0x7fc34c00eb80 comp rx=0 tx=0).stop 2023-03-03T06:26:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.450+0000 7fc35759e700 1 -- 172.21.15.3:0/52942282 shutdown_connections 2023-03-03T06:26:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.450+0000 7fc35759e700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc338064ee0 0x7fc338067390 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.450+0000 7fc35759e700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3581608d0 0x7fc3581f25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.450+0000 7fc35759e700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc358153670 0x7fc35814f070 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.450+0000 7fc35759e700 1 --2- 172.21.15.3:0/52942282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3580cfb60 0x7fc358152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.450+0000 7fc35759e700 1 -- 172.21.15.3:0/52942282 >> 172.21.15.3:0/52942282 conn(0x7fc35806e5c0 msgr2=0x7fc3580b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:52.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.450+0000 7fc35759e700 1 -- 172.21.15.3:0/52942282 shutdown_connections 2023-03-03T06:26:52.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.450+0000 7fc35759e700 1 -- 172.21.15.3:0/52942282 wait complete. 2023-03-03T06:26:52.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-03-03T06:26:52.497 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-03-03T06:26:52.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.690+0000 7f2752ee4700 1 Processor -- start 2023-03-03T06:26:52.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.690+0000 7f2752ee4700 1 -- start start 2023-03-03T06:26:52.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2752ee4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f274c15c3d0 0x7f274c15c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:52.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2752ee4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274c0cb770 0x7f274c0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:52.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2752ee4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f274c0c9df0 0x7f274c0ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:52.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2752ee4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274c0ca270 con 0x7f274c15c3d0 2023-03-03T06:26:52.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2752ee4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274c15d940 con 0x7f274c0c9df0 2023-03-03T06:26:52.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2752ee4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274c15cdc0 con 0x7f274c0cb770 2023-03-03T06:26:52.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2751ee2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f274c15c3d0 0x7f274c15c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:52.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2751ee2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f274c15c3d0 0x7f274c15c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52220/0 (socket says 172.21.15.3:52220) 2023-03-03T06:26:52.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2751ee2700 1 -- 172.21.15.3:0/2029991227 learned_addr learned my addr 172.21.15.3:0/2029991227 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f27516e1700 1 --2- 172.21.15.3:0/2029991227 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274c0cb770 0x7f274c0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f27526e3700 1 --2- 172.21.15.3:0/2029991227 >> v2:172.21.15.32:3300/0 conn(0x7f274c0c9df0 0x7f274c0ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:52.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2751ee2700 1 -- 172.21.15.3:0/2029991227 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274c0cb770 msgr2=0x7f274c0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:52.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2751ee2700 1 --2- 172.21.15.3:0/2029991227 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274c0cb770 0x7f274c0c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2751ee2700 1 -- 172.21.15.3:0/2029991227 >> v2:172.21.15.32:3300/0 conn(0x7f274c0c9df0 msgr2=0x7f274c0ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:52.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2751ee2700 1 --2- 172.21.15.3:0/2029991227 >> v2:172.21.15.32:3300/0 conn(0x7f274c0c9df0 0x7f274c0ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f2751ee2700 1 -- 172.21.15.3:0/2029991227 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c15d0a0 con 0x7f274c15c3d0 2023-03-03T06:26:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.694+0000 7f27526e3700 1 --2- 172.21.15.3:0/2029991227 >> v2:172.21.15.32:3300/0 conn(0x7f274c0c9df0 0x7f274c0ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.698+0000 7f2751ee2700 1 --2- 172.21.15.3:0/2029991227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f274c15c3d0 0x7f274c15c7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f274c0c9770 tx=0x7f2748009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd49dbb6c2cb9b60 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:52.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.698+0000 7f2750ee0700 1 -- 172.21.15.3:0/2029991227 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748011830 con 0x7f274c15c3d0 2023-03-03T06:26:52.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.698+0000 7f2750ee0700 1 -- 172.21.15.3:0/2029991227 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2748011990 con 0x7f274c15c3d0 2023-03-03T06:26:52.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.698+0000 7f2750ee0700 1 -- 172.21.15.3:0/2029991227 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748011d20 con 0x7f274c15c3d0 2023-03-03T06:26:52.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.698+0000 7f2752ee4700 1 -- 172.21.15.3:0/2029991227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f274c15c3d0 msgr2=0x7f274c15c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.698+0000 7f2752ee4700 1 --2- 172.21.15.3:0/2029991227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f274c15c3d0 0x7f274c15c7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f274c0c9770 tx=0x7f2748009bf0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.698+0000 7f2752ee4700 1 -- 172.21.15.3:0/2029991227 shutdown_connections 2023-03-03T06:26:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.698+0000 7f2752ee4700 1 --2- 172.21.15.3:0/2029991227 >> v2:172.21.15.32:3300/0 conn(0x7f274c0c9df0 0x7f274c0ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.698+0000 7f2752ee4700 1 --2- 172.21.15.3:0/2029991227 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274c0cb770 0x7f274c0c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.698+0000 7f2752ee4700 1 --2- 172.21.15.3:0/2029991227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f274c15c3d0 0x7f274c15c7a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.698+0000 7f2752ee4700 1 -- 172.21.15.3:0/2029991227 >> 172.21.15.3:0/2029991227 conn(0x7f274c070a30 msgr2=0x7f274c070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.698+0000 7f2752ee4700 1 -- 172.21.15.3:0/2029991227 shutdown_connections 2023-03-03T06:26:52.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.702+0000 7f2752ee4700 1 -- 172.21.15.3:0/2029991227 wait complete. 2023-03-03T06:26:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.702+0000 7f2752ee4700 1 Processor -- start 2023-03-03T06:26:52.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.702+0000 7f2752ee4700 1 -- start start 2023-03-03T06:26:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f2752ee4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f274c0c9df0 0x7f274c1515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f2752ee4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274c0cb770 0x7f274c15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f2752ee4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f274c15c3d0 0x7f274c06d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f2752ee4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c15b440 con 0x7f274c15c3d0 2023-03-03T06:26:52.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f2752ee4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c15b160 con 0x7f274c0c9df0 2023-03-03T06:26:52.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f2752ee4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0c23e0 con 0x7f274c0cb770 2023-03-03T06:26:52.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f27526e3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f274c15c3d0 0x7f274c06d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:52.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f27526e3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f274c15c3d0 0x7f274c06d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52224/0 (socket says 172.21.15.3:52224) 2023-03-03T06:26:52.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f27526e3700 1 -- 172.21.15.3:0/1118814030 learned_addr learned my addr 172.21.15.3:0/1118814030 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:52.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f2751ee2700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f274c0c9df0 0x7f274c1515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:52.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f27516e1700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274c0cb770 0x7f274c15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f27526e3700 1 -- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274c0cb770 msgr2=0x7f274c15a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f27526e3700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274c0cb770 0x7f274c15a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f27526e3700 1 -- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f274c0c9df0 msgr2=0x7f274c1515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f27526e3700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f274c0c9df0 0x7f274c1515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f27526e3700 1 -- 172.21.15.3:0/1118814030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2748009570 con 0x7f274c15c3d0 2023-03-03T06:26:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f27526e3700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f274c15c3d0 0x7f274c06d090 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2744007e10 tx=0x7f274400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f27427fc700 1 -- 172.21.15.3:0/1118814030 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274400dd20 con 0x7f274c15c3d0 2023-03-03T06:26:52.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f27427fc700 1 -- 172.21.15.3:0/1118814030 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274400f040 con 0x7f274c15c3d0 2023-03-03T06:26:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f27427fc700 1 -- 172.21.15.3:0/1118814030 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744003c80 con 0x7f274c15c3d0 2023-03-03T06:26:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f2752ee4700 1 -- 172.21.15.3:0/1118814030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274c15d660 con 0x7f274c15c3d0 2023-03-03T06:26:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.706+0000 7f2752ee4700 1 -- 172.21.15.3:0/1118814030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274c15d380 con 0x7f274c15c3d0 2023-03-03T06:26:52.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.710+0000 7f2752ee4700 1 -- 172.21.15.3:0/1118814030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274c0cbbe0 con 0x7f274c15c3d0 2023-03-03T06:26:52.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.710+0000 7f27427fc700 1 -- 172.21.15.3:0/1118814030 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f274401c030 con 0x7f274c15c3d0 2023-03-03T06:26:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.714+0000 7f27427fc700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2730064ee0 0x7f2730067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:52.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.714+0000 7f27427fc700 1 -- 172.21.15.3:0/1118814030 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f2744081c60 con 0x7f274c15c3d0 2023-03-03T06:26:52.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.714+0000 7f2751ee2700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2730064ee0 0x7f2730067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:52.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.718+0000 7f2751ee2700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2730064ee0 0x7f2730067390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2748000c00 tx=0x7f2748008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:52.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.718+0000 7f27427fc700 1 -- 172.21.15.3:0/1118814030 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f274404ff20 con 0x7f274c15c3d0 2023-03-03T06:26:52.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.954+0000 7f2752ee4700 1 -- 172.21.15.3:0/1118814030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f274c0cb150 con 0x7f274c15c3d0 2023-03-03T06:26:52.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.962+0000 7f27427fc700 1 -- 172.21.15.3:0/1118814030 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v13) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2744053540 con 0x7f274c15c3d0 2023-03-03T06:26:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.962+0000 7f2752ee4700 1 -- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2730064ee0 msgr2=0x7f2730067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.962+0000 7f2752ee4700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2730064ee0 0x7f2730067390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2748000c00 tx=0x7f2748008030 comp rx=0 tx=0).stop 2023-03-03T06:26:52.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.962+0000 7f2752ee4700 1 -- 172.21.15.3:0/1118814030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f274c15c3d0 msgr2=0x7f274c06d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:52.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.962+0000 7f2752ee4700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f274c15c3d0 0x7f274c06d090 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2744007e10 tx=0x7f274400d3b0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.962+0000 7f2752ee4700 1 -- 172.21.15.3:0/1118814030 shutdown_connections 2023-03-03T06:26:52.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.962+0000 7f2752ee4700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2730064ee0 0x7f2730067390 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.962+0000 7f2752ee4700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f274c15c3d0 0x7f274c06d090 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.962+0000 7f2752ee4700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f274c0cb770 0x7f274c15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.962+0000 7f2752ee4700 1 --2- 172.21.15.3:0/1118814030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f274c0c9df0 0x7f274c1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.962+0000 7f2752ee4700 1 -- 172.21.15.3:0/1118814030 >> 172.21.15.3:0/1118814030 conn(0x7f274c070a30 msgr2=0x7f274c0cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.962+0000 7f2752ee4700 1 -- 172.21.15.3:0/1118814030 shutdown_connections 2023-03-03T06:26:52.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:52.966+0000 7f2752ee4700 1 -- 172.21.15.3:0/1118814030 wait complete. 2023-03-03T06:26:53.004 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:53.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.202+0000 7f8460e76700 1 Processor -- start 2023-03-03T06:26:53.207 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.202+0000 7f8460e76700 1 -- start start 2023-03-03T06:26:53.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f8460e76700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f845c159230 0x7f845c0c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:53.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f8460e76700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f845c15d250 0x7f845c06c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:53.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f8460e76700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845c0ae150 con 0x7f845c158e60 2023-03-03T06:26:53.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f8460e76700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845c0b0e30 con 0x7f845c15d250 2023-03-03T06:26:53.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f8460e76700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845c06bf00 con 0x7f845c159230 2023-03-03T06:26:53.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f845bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f845c15d250 0x7f845c06c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:53.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f845bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f845c15d250 0x7f845c06c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52238/0 (socket says 172.21.15.3:52238) 2023-03-03T06:26:53.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f845bfff700 1 -- 172.21.15.3:0/227602756 learned_addr learned my addr 172.21.15.3:0/227602756 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:53.212 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f845affd700 1 --2- 172.21.15.3:0/227602756 >> v2:172.21.15.32:3300/0 conn(0x7f845c159230 0x7f845c0c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:53.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f845bfff700 1 -- 172.21.15.3:0/227602756 >> v2:172.21.15.32:3300/0 conn(0x7f845c159230 msgr2=0x7f845c0c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:53.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f845bfff700 1 --2- 172.21.15.3:0/227602756 >> v2:172.21.15.32:3300/0 conn(0x7f845c159230 0x7f845c0c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f845bfff700 1 -- 172.21.15.3:0/227602756 >> v1:172.21.15.32:6789/0 conn(0x7f845c158e60 legacy=0x7f845c15cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:53.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f845bfff700 1 -- 172.21.15.3:0/227602756 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c06c4c0 con 0x7f845c15d250 2023-03-03T06:26:53.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f845bfff700 1 --2- 172.21.15.3:0/227602756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f845c15d250 0x7f845c06c950 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f845c0cc1a0 tx=0x7f845400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bce0d0bda3e3dd3 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:53.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.206+0000 7f845a7fc700 1 -- 172.21.15.3:0/227602756 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845400e070 con 0x7f845c15d250 2023-03-03T06:26:53.214 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.210+0000 7f845a7fc700 1 -- 172.21.15.3:0/227602756 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845400b910 con 0x7f845c15d250 2023-03-03T06:26:53.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.210+0000 7f845a7fc700 1 -- 172.21.15.3:0/227602756 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845400bcc0 con 0x7f845c15d250 2023-03-03T06:26:53.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.210+0000 7f8460e76700 1 -- 172.21.15.3:0/227602756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f845c15d250 msgr2=0x7f845c06c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:53.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.210+0000 7f8460e76700 1 --2- 172.21.15.3:0/227602756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f845c15d250 0x7f845c06c950 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f845c0cc1a0 tx=0x7f845400b330 comp rx=0 tx=0).stop 2023-03-03T06:26:53.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.210+0000 7f8460e76700 1 -- 172.21.15.3:0/227602756 shutdown_connections 2023-03-03T06:26:53.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.210+0000 7f8460e76700 1 --2- 172.21.15.3:0/227602756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f845c15d250 0x7f845c06c950 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.210+0000 7f8460e76700 1 --2- 172.21.15.3:0/227602756 >> v2:172.21.15.32:3300/0 conn(0x7f845c159230 0x7f845c0c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.210+0000 7f8460e76700 1 -- 172.21.15.3:0/227602756 >> 172.21.15.3:0/227602756 conn(0x7f845c0cf370 msgr2=0x7f845c0cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:53.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.210+0000 7f8460e76700 1 -- 172.21.15.3:0/227602756 shutdown_connections 2023-03-03T06:26:53.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.210+0000 7f8460e76700 1 -- 172.21.15.3:0/227602756 wait complete. 2023-03-03T06:26:53.217 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f8460e76700 1 Processor -- start 2023-03-03T06:26:53.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f8460e76700 1 -- start start 2023-03-03T06:26:53.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f8460e76700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f845c158e60 0x7f845c0b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:53.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f8460e76700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f845c159230 0x7f845c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:53.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f8460e76700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f845c15d250 0x7f845c0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:53.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f8460e76700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c0c8850 con 0x7f845c158e60 2023-03-03T06:26:53.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f8460e76700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c0b4590 con 0x7f845c15d250 2023-03-03T06:26:53.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f8460e76700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c0c7bd0 con 0x7f845c159230 2023-03-03T06:26:53.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f845b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f845c158e60 0x7f845c0b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:53.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f845b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f845c158e60 0x7f845c0b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52248/0 (socket says 172.21.15.3:52248) 2023-03-03T06:26:53.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f845b7fe700 1 -- 172.21.15.3:0/3152933328 learned_addr learned my addr 172.21.15.3:0/3152933328 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:53.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f845bfff700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f845c15d250 0x7f845c0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:53.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f845affd700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f845c159230 0x7f845c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:53.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f845b7fe700 1 -- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f845c159230 msgr2=0x7f845c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:53.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f845b7fe700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f845c159230 0x7f845c0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f845b7fe700 1 -- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f845c15d250 msgr2=0x7f845c0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:53.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f845b7fe700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f845c15d250 0x7f845c0cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.214+0000 7f845b7fe700 1 -- 172.21.15.3:0/3152933328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8454009b30 con 0x7f845c158e60 2023-03-03T06:26:53.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.218+0000 7f845b7fe700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f845c158e60 0x7f845c0b4050 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f844c00c500 tx=0x7f844c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:53.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.218+0000 7f8443fff700 1 -- 172.21.15.3:0/3152933328 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c0052d0 con 0x7f845c158e60 2023-03-03T06:26:53.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.218+0000 7f8443fff700 1 -- 172.21.15.3:0/3152933328 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c005430 con 0x7f845c158e60 2023-03-03T06:26:53.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.218+0000 7f8460e76700 1 -- 172.21.15.3:0/3152933328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845c0ce360 con 0x7f845c158e60 2023-03-03T06:26:53.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.218+0000 7f8443fff700 1 -- 172.21.15.3:0/3152933328 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c0055a0 con 0x7f845c158e60 2023-03-03T06:26:53.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.218+0000 7f8460e76700 1 -- 172.21.15.3:0/3152933328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845c0ce080 con 0x7f845c158e60 2023-03-03T06:26:53.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.218+0000 7f8460e76700 1 -- 172.21.15.3:0/3152933328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845c06c150 con 0x7f845c158e60 2023-03-03T06:26:53.227 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.222+0000 7f8443fff700 1 -- 172.21.15.3:0/3152933328 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f844c019030 con 0x7f845c158e60 2023-03-03T06:26:53.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.222+0000 7f8443fff700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8448064ee0 0x7f8448067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:53.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.222+0000 7f8443fff700 1 -- 172.21.15.3:0/3152933328 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f844c080f80 con 0x7f845c158e60 2023-03-03T06:26:53.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.222+0000 7f845affd700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8448064ee0 0x7f8448067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:53.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.226+0000 7f845affd700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8448064ee0 0x7f8448067390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f845c152000 tx=0x7f8450009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:53.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.230+0000 7f8443fff700 1 -- 172.21.15.3:0/3152933328 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f844c04f1e0 con 0x7f845c158e60 2023-03-03T06:26:53.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.410+0000 7f8460e76700 1 -- 172.21.15.3:0/3152933328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f845c0c8990 con 0x7f845c158e60 2023-03-03T06:26:53.419 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.414+0000 7f8443fff700 1 -- 172.21.15.3:0/3152933328 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+14280 (secure 0 0 0) 0x7f845c0c8990 con 0x7f845c158e60 2023-03-03T06:26:53.420 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:53.420 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":39,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:26:49.273481+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:53.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.418+0000 7f8460e76700 1 -- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8448064ee0 msgr2=0x7f8448067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.418+0000 7f8460e76700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8448064ee0 0x7f8448067390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f845c152000 tx=0x7f8450009240 comp rx=0 tx=0).stop 2023-03-03T06:26:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.418+0000 7f8460e76700 1 -- 172.21.15.3:0/3152933328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f845c158e60 msgr2=0x7f845c0b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.418+0000 7f8460e76700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f845c158e60 0x7f845c0b4050 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f844c00c500 tx=0x7f844c00c810 comp rx=0 tx=0).stop 2023-03-03T06:26:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.418+0000 7f8460e76700 1 -- 172.21.15.3:0/3152933328 shutdown_connections 2023-03-03T06:26:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.418+0000 7f8460e76700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8448064ee0 0x7f8448067390 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.418+0000 7f8460e76700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f845c15d250 0x7f845c0cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.418+0000 7f8460e76700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f845c159230 0x7f845c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.418+0000 7f8460e76700 1 --2- 172.21.15.3:0/3152933328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f845c158e60 0x7f845c0b4050 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.418+0000 7f8460e76700 1 -- 172.21.15.3:0/3152933328 >> 172.21.15.3:0/3152933328 conn(0x7f845c0cf370 msgr2=0x7f845c153350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:53.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.418+0000 7f8460e76700 1 -- 172.21.15.3:0/3152933328 shutdown_connections 2023-03-03T06:26:53.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.418+0000 7f8460e76700 1 -- 172.21.15.3:0/3152933328 wait complete. 2023-03-03T06:26:53.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-03T06:25:00.514768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-03T06:25:05.271759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-03-03T06:26:53.460 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-03T06:26:53.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.666+0000 7fc3a8c66700 1 Processor -- start 2023-03-03T06:26:53.673 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.666+0000 7fc3a8c66700 1 -- start start 2023-03-03T06:26:53.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.670+0000 7fc3a8c66700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3a4156ba0 0x7fc3a4071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:53.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.670+0000 7fc3a8c66700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc3a415d6e0 0x7fc3a40cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:53.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.670+0000 7fc3a8c66700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a40cf3b0 con 0x7fc3a40cfd80 2023-03-03T06:26:53.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.670+0000 7fc3a8c66700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a4156f70 con 0x7fc3a415d6e0 2023-03-03T06:26:53.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.670+0000 7fc3a8c66700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40cd320 con 0x7fc3a4156ba0 2023-03-03T06:26:53.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.670+0000 7fc3a3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc3a40cfd80 0x7fc3a40d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47336/0 (socket says 172.21.15.3:47336) 2023-03-03T06:26:53.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.670+0000 7fc3a3fff700 1 -- 172.21.15.3:0/3882259806 learned_addr learned my addr 172.21.15.3:0/3882259806 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:53.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.670+0000 7fc3a37fe700 1 --2- 172.21.15.3:0/3882259806 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3a4156ba0 0x7fc3a4071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:53.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.670+0000 7fc3a2ffd700 1 --2- 172.21.15.3:0/3882259806 >> v2:172.21.15.32:3300/0 conn(0x7fc3a415d6e0 0x7fc3a40cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.674+0000 7fc3a27fc700 1 -- 172.21.15.3:0/3882259806 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242757392 0 0) 0x7fc3a40cf3b0 con 0x7fc3a40cfd80 2023-03-03T06:26:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.674+0000 7fc3a27fc700 1 -- 172.21.15.3:0/3882259806 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc38c0035d0 con 0x7fc3a40cfd80 2023-03-03T06:26:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.674+0000 7fc3a37fe700 1 -- 172.21.15.3:0/3882259806 >> v2:172.21.15.32:3300/0 conn(0x7fc3a415d6e0 msgr2=0x7fc3a40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.674+0000 7fc3a37fe700 1 --2- 172.21.15.3:0/3882259806 >> v2:172.21.15.32:3300/0 conn(0x7fc3a415d6e0 0x7fc3a40cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.674+0000 7fc3a37fe700 1 -- 172.21.15.3:0/3882259806 >> v1:172.21.15.3:6789/0 conn(0x7fc3a40cfd80 legacy=0x7fc3a40d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.674+0000 7fc3a37fe700 1 -- 172.21.15.3:0/3882259806 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a40cd600 con 0x7fc3a4156ba0 2023-03-03T06:26:53.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.674+0000 7fc3a2ffd700 1 --2- 172.21.15.3:0/3882259806 >> v2:172.21.15.32:3300/0 conn(0x7fc3a415d6e0 0x7fc3a40cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:53.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.674+0000 7fc3a37fe700 1 --2- 172.21.15.3:0/3882259806 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3a4156ba0 0x7fc3a4071280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc394009d90 tx=0x7fc39400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f8748413728780 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:53.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.674+0000 7fc3a27fc700 1 -- 172.21.15.3:0/3882259806 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39400e070 con 0x7fc3a4156ba0 2023-03-03T06:26:53.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.674+0000 7fc3a27fc700 1 -- 172.21.15.3:0/3882259806 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39400bb20 con 0x7fc3a4156ba0 2023-03-03T06:26:53.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.674+0000 7fc3a27fc700 1 -- 172.21.15.3:0/3882259806 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394012430 con 0x7fc3a4156ba0 2023-03-03T06:26:53.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.678+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3882259806 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3a4156ba0 msgr2=0x7fc3a4071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:53.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.678+0000 7fc3a8c66700 1 --2- 172.21.15.3:0/3882259806 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3a4156ba0 0x7fc3a4071280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc394009d90 tx=0x7fc39400b330 comp rx=0 tx=0).stop 2023-03-03T06:26:53.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.678+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3882259806 shutdown_connections 2023-03-03T06:26:53.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.678+0000 7fc3a8c66700 1 --2- 172.21.15.3:0/3882259806 >> v2:172.21.15.32:3300/0 conn(0x7fc3a415d6e0 0x7fc3a40cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.678+0000 7fc3a8c66700 1 --2- 172.21.15.3:0/3882259806 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3a4156ba0 0x7fc3a4071280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.678+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3882259806 >> 172.21.15.3:0/3882259806 conn(0x7fc3a406e5c0 msgr2=0x7fc3a406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.678+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3882259806 shutdown_connections 2023-03-03T06:26:53.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.678+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3882259806 wait complete. 2023-03-03T06:26:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.678+0000 7fc3a8c66700 1 Processor -- start 2023-03-03T06:26:53.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.678+0000 7fc3a8c66700 1 -- start start 2023-03-03T06:26:53.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a8c66700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3a40cfd80 0x7fc3a40b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:53.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a8c66700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3a4156ba0 0x7fc3a40cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:53.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a8c66700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a415d6e0 0x7fc3a40b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:53.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a8c66700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40cba60 con 0x7fc3a40cfd80 2023-03-03T06:26:53.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a8c66700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40cb3c0 con 0x7fc3a415d6e0 2023-03-03T06:26:53.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a8c66700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a41f5e60 con 0x7fc3a4156ba0 2023-03-03T06:26:53.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3a40cfd80 0x7fc3a40b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:53.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3a40cfd80 0x7fc3a40b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52252/0 (socket says 172.21.15.3:52252) 2023-03-03T06:26:53.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a37fe700 1 -- 172.21.15.3:0/3248315157 learned_addr learned my addr 172.21.15.3:0/3248315157 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:53.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a3fff700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a415d6e0 0x7fc3a40b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:53.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a2ffd700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3a4156ba0 0x7fc3a40cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:53.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a37fe700 1 -- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3a4156ba0 msgr2=0x7fc3a40cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:53.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a37fe700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3a4156ba0 0x7fc3a40cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a37fe700 1 -- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a415d6e0 msgr2=0x7fc3a40b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:53.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a37fe700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a415d6e0 0x7fc3a40b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a37fe700 1 -- 172.21.15.3:0/3248315157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3980094b0 con 0x7fc3a40cfd80 2023-03-03T06:26:53.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a2ffd700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3a4156ba0 0x7fc3a40cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:53.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a3fff700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a415d6e0 0x7fc3a40b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:53.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a37fe700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3a40cfd80 0x7fc3a40b3e50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc394014040 tx=0x7fc394004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:53.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc38bfff700 1 -- 172.21.15.3:0/3248315157 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394012430 con 0x7fc3a40cfd80 2023-03-03T06:26:53.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc38bfff700 1 -- 172.21.15.3:0/3248315157 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc394020070 con 0x7fc3a40cfd80 2023-03-03T06:26:53.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.682+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3248315157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc394009a70 con 0x7fc3a40cfd80 2023-03-03T06:26:53.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.686+0000 7fc38bfff700 1 -- 172.21.15.3:0/3248315157 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394004280 con 0x7fc3a40cfd80 2023-03-03T06:26:53.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.686+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3248315157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a40717c0 con 0x7fc3a40cfd80 2023-03-03T06:26:53.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.686+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3248315157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a40cd8e0 con 0x7fc3a40cfd80 2023-03-03T06:26:53.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.686+0000 7fc38bfff700 1 -- 172.21.15.3:0/3248315157 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc39402e070 con 0x7fc3a40cfd80 2023-03-03T06:26:53.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.690+0000 7fc38bfff700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc390064ee0 0x7fc390067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:53.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.690+0000 7fc38bfff700 1 -- 172.21.15.3:0/3248315157 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fc394022070 con 0x7fc3a40cfd80 2023-03-03T06:26:53.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.690+0000 7fc3a2ffd700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc390064ee0 0x7fc390067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:53.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.690+0000 7fc3a2ffd700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc390064ee0 0x7fc390067390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc3a415ee20 tx=0x7fc398008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:53.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.698+0000 7fc38bfff700 1 -- 172.21.15.3:0/3248315157 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc394051d20 con 0x7fc3a40cfd80 2023-03-03T06:26:53.883 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.878+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3248315157 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3a40cef40 con 0x7fc3a40cfd80 2023-03-03T06:26:53.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.878+0000 7fc38bfff700 1 -- 172.21.15.3:0/3248315157 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc39402c020 con 0x7fc3a40cfd80 2023-03-03T06:26:53.885 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2023-03-03T06:26:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.882+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc390064ee0 msgr2=0x7fc390067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.882+0000 7fc3a8c66700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc390064ee0 0x7fc390067390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc3a415ee20 tx=0x7fc398008040 comp rx=0 tx=0).stop 2023-03-03T06:26:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.882+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3248315157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3a40cfd80 msgr2=0x7fc3a40b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.886+0000 7fc3a8c66700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3a40cfd80 0x7fc3a40b3e50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc394014040 tx=0x7fc394004af0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.886+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3248315157 shutdown_connections 2023-03-03T06:26:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.886+0000 7fc3a8c66700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc390064ee0 0x7fc390067390 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.886+0000 7fc3a8c66700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3a415d6e0 0x7fc3a40b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.886+0000 7fc3a8c66700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc3a4156ba0 0x7fc3a40cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.886+0000 7fc3a8c66700 1 --2- 172.21.15.3:0/3248315157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3a40cfd80 0x7fc3a40b3e50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:53.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.886+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3248315157 >> 172.21.15.3:0/3248315157 conn(0x7fc3a406e5c0 msgr2=0x7fc3a40b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.886+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3248315157 shutdown_connections 2023-03-03T06:26:53.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:53.886+0000 7fc3a8c66700 1 -- 172.21.15.3:0/3248315157 wait complete. 2023-03-03T06:26:53.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-03T06:26:54.131 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.126+0000 7f3e089e0700 1 Processor -- start 2023-03-03T06:26:54.132 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.126+0000 7f3e089e0700 1 -- start start 2023-03-03T06:26:54.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.130+0000 7f3e089e0700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3e040cee10 0x7f3e0406c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.130+0000 7f3e089e0700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e0406d790 con 0x7f3e04159220 2023-03-03T06:26:54.137 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.130+0000 7f3e089e0700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e0406d1d0 con 0x7f3e0406c950 2023-03-03T06:26:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.130+0000 7f3e02d9d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3e04159220 0x7f3e040ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47350/0 (socket says 172.21.15.3:47350) 2023-03-03T06:26:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.130+0000 7f3e02d9d700 1 -- 172.21.15.3:0/3692116860 learned_addr learned my addr 172.21.15.3:0/3692116860 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.130+0000 7f3e089e0700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0415a600 con 0x7f3e040cee10 2023-03-03T06:26:54.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.130+0000 7f3e0259c700 1 --2- 172.21.15.3:0/3692116860 >> v2:172.21.15.32:3300/0 conn(0x7f3e040cee10 0x7f3e0406c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.134+0000 7f3e01d9b700 1 -- 172.21.15.3:0/3692116860 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894028324 0 0) 0x7f3e0406d1d0 con 0x7f3e0406c950 2023-03-03T06:26:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.134+0000 7f3e01d9b700 1 -- 172.21.15.3:0/3692116860 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dec0035d0 con 0x7f3e0406c950 2023-03-03T06:26:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.134+0000 7f3e01d9b700 1 -- 172.21.15.3:0/3692116860 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588963623 0 0) 0x7f3e0406d790 con 0x7f3e04159220 2023-03-03T06:26:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.134+0000 7f3e01d9b700 1 -- 172.21.15.3:0/3692116860 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e0406d1d0 con 0x7f3e04159220 2023-03-03T06:26:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.134+0000 7f3e0259c700 1 -- 172.21.15.3:0/3692116860 >> v1:172.21.15.32:6789/0 conn(0x7f3e0406c950 legacy=0x7f3e040cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:54.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.134+0000 7f3e0259c700 1 -- 172.21.15.3:0/3692116860 >> v1:172.21.15.3:6789/0 conn(0x7f3e04159220 legacy=0x7f3e040ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:54.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.134+0000 7f3e0259c700 1 -- 172.21.15.3:0/3692116860 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0406d4b0 con 0x7f3e040cee10 2023-03-03T06:26:54.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.134+0000 7f3e0259c700 1 --2- 172.21.15.3:0/3692116860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e040cee10 0x7f3e0406c410 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009850 tx=0x7f3df8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2818f6d75d880b93 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:54.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.134+0000 7f3e01d9b700 1 -- 172.21.15.3:0/3692116860 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8011830 con 0x7f3e040cee10 2023-03-03T06:26:54.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.134+0000 7f3e01d9b700 1 -- 172.21.15.3:0/3692116860 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df8011990 con 0x7f3e040cee10 2023-03-03T06:26:54.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.134+0000 7f3e01d9b700 1 -- 172.21.15.3:0/3692116860 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8011d40 con 0x7f3e040cee10 2023-03-03T06:26:54.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.138+0000 7f3e089e0700 1 -- 172.21.15.3:0/3692116860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e040cee10 msgr2=0x7f3e0406c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:54.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.138+0000 7f3e089e0700 1 --2- 172.21.15.3:0/3692116860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e040cee10 0x7f3e0406c410 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009850 tx=0x7f3df8009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:54.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.138+0000 7f3e089e0700 1 -- 172.21.15.3:0/3692116860 shutdown_connections 2023-03-03T06:26:54.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.138+0000 7f3e089e0700 1 --2- 172.21.15.3:0/3692116860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e040cee10 0x7f3e0406c410 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.138+0000 7f3e089e0700 1 -- 172.21.15.3:0/3692116860 >> 172.21.15.3:0/3692116860 conn(0x7f3e040607c0 msgr2=0x7f3e04060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:54.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.138+0000 7f3e089e0700 1 -- 172.21.15.3:0/3692116860 shutdown_connections 2023-03-03T06:26:54.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.138+0000 7f3e089e0700 1 -- 172.21.15.3:0/3692116860 wait complete. 2023-03-03T06:26:54.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.142+0000 7f3e089e0700 1 Processor -- start 2023-03-03T06:26:54.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.142+0000 7f3e089e0700 1 -- start start 2023-03-03T06:26:54.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.142+0000 7f3e089e0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e0406c950 0x7f3e04147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:54.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.142+0000 7f3e089e0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e040cee10 0x7f3e040c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:54.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.142+0000 7f3e089e0700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3e04159220 0x7f3e040c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:54.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.142+0000 7f3e089e0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e040c5880 con 0x7f3e040cee10 2023-03-03T06:26:54.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.142+0000 7f3e089e0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e040c4d70 con 0x7f3e0406c950 2023-03-03T06:26:54.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.142+0000 7f3e089e0700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e040c5050 con 0x7f3e04159220 2023-03-03T06:26:54.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.142+0000 7f3e0259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e040cee10 0x7f3e040c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:54.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.142+0000 7f3e0259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e040cee10 0x7f3e040c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52264/0 (socket says 172.21.15.3:52264) 2023-03-03T06:26:54.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.142+0000 7f3e0259c700 1 -- 172.21.15.3:0/532435401 learned_addr learned my addr 172.21.15.3:0/532435401 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:54.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.142+0000 7f3e02d9d700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e0406c950 0x7f3e04147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:54.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.142+0000 7f3e0359e700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3e04159220 0x7f3e040c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:54.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.146+0000 7f3e0259c700 1 -- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3e04159220 msgr2=0x7f3e040c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:54.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.146+0000 7f3e0259c700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3e04159220 0x7f3e040c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.146+0000 7f3e0259c700 1 -- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e0406c950 msgr2=0x7f3e04147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:54.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.146+0000 7f3e0259c700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e0406c950 0x7f3e04147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.146+0000 7f3e0259c700 1 -- 172.21.15.3:0/532435401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df80094b0 con 0x7f3e040cee10 2023-03-03T06:26:54.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.146+0000 7f3e0259c700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e040cee10 0x7f3e040c41f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3df80090b0 tx=0x7f3df8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:54.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.146+0000 7f3deb7fe700 1 -- 172.21.15.3:0/532435401 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8019a00 con 0x7f3e040cee10 2023-03-03T06:26:54.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.146+0000 7f3deb7fe700 1 -- 172.21.15.3:0/532435401 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df801e070 con 0x7f3e040cee10 2023-03-03T06:26:54.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.146+0000 7f3deb7fe700 1 -- 172.21.15.3:0/532435401 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8019dc0 con 0x7f3e040cee10 2023-03-03T06:26:54.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.146+0000 7f3e089e0700 1 -- 172.21.15.3:0/532435401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e040cc790 con 0x7f3e040cee10 2023-03-03T06:26:54.153 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.146+0000 7f3e089e0700 1 -- 172.21.15.3:0/532435401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e040cc4b0 con 0x7f3e040cee10 2023-03-03T06:26:54.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.150+0000 7f3e089e0700 1 -- 172.21.15.3:0/532435401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e0415a7c0 con 0x7f3e040cee10 2023-03-03T06:26:54.155 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.150+0000 7f3deb7fe700 1 -- 172.21.15.3:0/532435401 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3df800f050 con 0x7f3e040cee10 2023-03-03T06:26:54.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.150+0000 7f3deb7fe700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3df0064ee0 0x7f3df0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:54.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.150+0000 7f3deb7fe700 1 -- 172.21.15.3:0/532435401 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3df8082c70 con 0x7f3e040cee10 2023-03-03T06:26:54.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.154+0000 7f3e02d9d700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3df0064ee0 0x7f3df0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:54.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.154+0000 7f3e02d9d700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3df0064ee0 0x7f3df0067390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3df4007fd0 tx=0x7f3df4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:54.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.158+0000 7f3deb7fe700 1 -- 172.21.15.3:0/532435401 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f3df8050f30 con 0x7f3e040cee10 2023-03-03T06:26:54.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.334+0000 7f3e089e0700 1 -- 172.21.15.3:0/532435401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3e040ad6c0 con 0x7f3e040cee10 2023-03-03T06:26:54.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.338+0000 7f3deb7fe700 1 -- 172.21.15.3:0/532435401 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3df80200a0 con 0x7f3e040cee10 2023-03-03T06:26:54.342 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 8 2023-03-03T06:26:54.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.338+0000 7f3e089e0700 1 -- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3df0064ee0 msgr2=0x7f3df0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:54.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.338+0000 7f3e089e0700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3df0064ee0 0x7f3df0067390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3df4007fd0 tx=0x7f3df4009040 comp rx=0 tx=0).stop 2023-03-03T06:26:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.338+0000 7f3e089e0700 1 -- 172.21.15.3:0/532435401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e040cee10 msgr2=0x7f3e040c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.338+0000 7f3e089e0700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e040cee10 0x7f3e040c41f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3df80090b0 tx=0x7f3df8004770 comp rx=0 tx=0).stop 2023-03-03T06:26:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.338+0000 7f3e089e0700 1 -- 172.21.15.3:0/532435401 shutdown_connections 2023-03-03T06:26:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.338+0000 7f3e089e0700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3df0064ee0 0x7f3df0067390 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.338+0000 7f3e089e0700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3e04159220 0x7f3e040c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.338+0000 7f3e089e0700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e040cee10 0x7f3e040c41f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.338+0000 7f3e089e0700 1 --2- 172.21.15.3:0/532435401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e0406c950 0x7f3e04147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.338+0000 7f3e089e0700 1 -- 172.21.15.3:0/532435401 >> 172.21.15.3:0/532435401 conn(0x7f3e040607c0 msgr2=0x7f3e040afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.342+0000 7f3e089e0700 1 -- 172.21.15.3:0/532435401 shutdown_connections 2023-03-03T06:26:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.342+0000 7f3e089e0700 1 -- 172.21.15.3:0/532435401 wait complete. 2023-03-03T06:26:54.382 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-03-03T06:26:54.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.582+0000 7f52d30fd700 1 Processor -- start 2023-03-03T06:26:54.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.582+0000 7f52d30fd700 1 -- start start 2023-03-03T06:26:54.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d30fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52cc0cde60 0x7f52cc0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:54.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d30fd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52cc0ce550 0x7f52cc0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:54.590 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d30fd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc15d8a0 con 0x7f52cc159100 2023-03-03T06:26:54.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d30fd700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc0cb010 con 0x7f52cc0ce550 2023-03-03T06:26:54.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d30fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0cef10 con 0x7f52cc0cde60 2023-03-03T06:26:54.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d18fa700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52cc0cde60 0x7f52cc0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:54.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d20fb700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f52cc159100 0x7f52cc0ae1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:57918/0 (socket says 172.21.15.3:57918) 2023-03-03T06:26:54.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d20fb700 1 -- 172.21.15.3:0/2729811023 learned_addr learned my addr 172.21.15.3:0/2729811023 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:54.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d28fc700 1 --2- 172.21.15.3:0/2729811023 >> v2:172.21.15.32:3300/0 conn(0x7f52cc0ce550 0x7f52cc0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:54.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d18fa700 1 -- 172.21.15.3:0/2729811023 >> v2:172.21.15.32:3300/0 conn(0x7f52cc0ce550 msgr2=0x7f52cc0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:54.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d18fa700 1 --2- 172.21.15.3:0/2729811023 >> v2:172.21.15.32:3300/0 conn(0x7f52cc0ce550 0x7f52cc0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d18fa700 1 -- 172.21.15.3:0/2729811023 >> v1:172.21.15.32:6789/0 conn(0x7f52cc159100 legacy=0x7f52cc0ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:54.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d18fa700 1 -- 172.21.15.3:0/2729811023 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc0caa50 con 0x7f52cc0cde60 2023-03-03T06:26:54.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d18fa700 1 --2- 172.21.15.3:0/2729811023 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52cc0cde60 0x7f52cc0afb60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009850 tx=0x7f52bc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=515f31c3bbfd08b9 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:54.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d10f9700 1 -- 172.21.15.3:0/2729811023 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00f8a0 con 0x7f52cc0cde60 2023-03-03T06:26:54.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d10f9700 1 -- 172.21.15.3:0/2729811023 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc00fa00 con 0x7f52cc0cde60 2023-03-03T06:26:54.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.586+0000 7f52d10f9700 1 -- 172.21.15.3:0/2729811023 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00fd90 con 0x7f52cc0cde60 2023-03-03T06:26:54.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.590+0000 7f52d30fd700 1 -- 172.21.15.3:0/2729811023 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52cc0cde60 msgr2=0x7f52cc0afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:54.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.590+0000 7f52d30fd700 1 --2- 172.21.15.3:0/2729811023 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52cc0cde60 0x7f52cc0afb60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009850 tx=0x7f52bc009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:54.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.590+0000 7f52d30fd700 1 -- 172.21.15.3:0/2729811023 shutdown_connections 2023-03-03T06:26:54.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.590+0000 7f52d30fd700 1 --2- 172.21.15.3:0/2729811023 >> v2:172.21.15.32:3300/0 conn(0x7f52cc0ce550 0x7f52cc0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.590+0000 7f52d30fd700 1 --2- 172.21.15.3:0/2729811023 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52cc0cde60 0x7f52cc0afb60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.590+0000 7f52d30fd700 1 -- 172.21.15.3:0/2729811023 >> 172.21.15.3:0/2729811023 conn(0x7f52cc070a30 msgr2=0x7f52cc070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:54.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.590+0000 7f52d30fd700 1 -- 172.21.15.3:0/2729811023 shutdown_connections 2023-03-03T06:26:54.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.590+0000 7f52d30fd700 1 -- 172.21.15.3:0/2729811023 wait complete. 2023-03-03T06:26:54.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.594+0000 7f52d30fd700 1 Processor -- start 2023-03-03T06:26:54.601 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.594+0000 7f52d30fd700 1 -- start start 2023-03-03T06:26:54.602 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d30fd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52cc0cde60 0x7f52cc1f0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:54.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d30fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52cc0ce550 0x7f52cc1ed0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:54.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d30fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc159100 0x7f52cc149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:54.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d30fd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc149af0 con 0x7f52cc0cde60 2023-03-03T06:26:54.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d30fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc1f13c0 con 0x7f52cc159100 2023-03-03T06:26:54.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d30fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc149730 con 0x7f52cc0ce550 2023-03-03T06:26:54.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d20fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52cc0cde60 0x7f52cc1f0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:54.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d20fb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52cc0cde60 0x7f52cc1f0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52276/0 (socket says 172.21.15.3:52276) 2023-03-03T06:26:54.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d20fb700 1 -- 172.21.15.3:0/641350918 learned_addr learned my addr 172.21.15.3:0/641350918 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:54.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d18fa700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52cc0ce550 0x7f52cc1ed0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:54.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d28fc700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc159100 0x7f52cc149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:54.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d20fb700 1 -- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52cc0ce550 msgr2=0x7f52cc1ed0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:54.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d20fb700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52cc0ce550 0x7f52cc1ed0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d20fb700 1 -- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc159100 msgr2=0x7f52cc149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:54.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d20fb700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc159100 0x7f52cc149f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d20fb700 1 -- 172.21.15.3:0/641350918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc0094b0 con 0x7f52cc0cde60 2023-03-03T06:26:54.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d20fb700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52cc0cde60 0x7f52cc1f0e80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f52c800cba0 tx=0x7f52c800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:54.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52c2ffd700 1 -- 172.21.15.3:0/641350918 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c800a9e0 con 0x7f52cc0cde60 2023-03-03T06:26:54.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52c2ffd700 1 -- 172.21.15.3:0/641350918 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c800ab40 con 0x7f52cc0cde60 2023-03-03T06:26:54.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d30fd700 1 -- 172.21.15.3:0/641350918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52cc15db60 con 0x7f52cc0cde60 2023-03-03T06:26:54.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52c2ffd700 1 -- 172.21.15.3:0/641350918 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8012400 con 0x7f52cc0cde60 2023-03-03T06:26:54.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.598+0000 7f52d30fd700 1 -- 172.21.15.3:0/641350918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc0cad30 con 0x7f52cc0cde60 2023-03-03T06:26:54.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.602+0000 7f52d30fd700 1 -- 172.21.15.3:0/641350918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52cc1594d0 con 0x7f52cc0cde60 2023-03-03T06:26:54.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.606+0000 7f52c2ffd700 1 -- 172.21.15.3:0/641350918 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f52c8019030 con 0x7f52cc0cde60 2023-03-03T06:26:54.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.606+0000 7f52c2ffd700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f52b0064ee0 0x7f52b0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:54.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.606+0000 7f52c2ffd700 1 -- 172.21.15.3:0/641350918 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f52c80815b0 con 0x7f52cc0cde60 2023-03-03T06:26:54.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.606+0000 7f52d18fa700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f52b0064ee0 0x7f52b0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:54.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.610+0000 7f52d18fa700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f52b0064ee0 0x7f52b0067390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f52bc000c00 tx=0x7f52bc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:54.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.610+0000 7f52c2ffd700 1 -- 172.21.15.3:0/641350918 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f52c804a400 con 0x7f52cc0cde60 2023-03-03T06:26:54.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.794+0000 7f52d30fd700 1 -- 172.21.15.3:0/641350918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f52cc073e90 con 0x7f52cc0cde60 2023-03-03T06:26:54.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.794+0000 7f52c2ffd700 1 -- 172.21.15.3:0/641350918 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) v1 ==== 84+0+126 (secure 0 0 0) 0x7f52c804f920 con 0x7f52cc0cde60 2023-03-03T06:26:54.801 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:54.801 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-03-03T06:26:54.801 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:54.801 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2023-03-03T06:26:54.802 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2023-03-03T06:26:54.802 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2023-03-03T06:26:54.802 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:26:54.802 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2023-03-03T06:26:54.802 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2023-03-03T06:26:54.802 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:26:54.802 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-03-03T06:26:54.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.798+0000 7f52d30fd700 1 -- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f52b0064ee0 msgr2=0x7f52b0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.798+0000 7f52d30fd700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f52b0064ee0 0x7f52b0067390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f52bc000c00 tx=0x7f52bc008030 comp rx=0 tx=0).stop 2023-03-03T06:26:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.798+0000 7f52d30fd700 1 -- 172.21.15.3:0/641350918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52cc0cde60 msgr2=0x7f52cc1f0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.798+0000 7f52d30fd700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52cc0cde60 0x7f52cc1f0e80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f52c800cba0 tx=0x7f52c800cc80 comp rx=0 tx=0).stop 2023-03-03T06:26:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.798+0000 7f52d30fd700 1 -- 172.21.15.3:0/641350918 shutdown_connections 2023-03-03T06:26:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.798+0000 7f52d30fd700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f52b0064ee0 0x7f52b0067390 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.798+0000 7f52d30fd700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52cc159100 0x7f52cc149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.798+0000 7f52d30fd700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f52cc0ce550 0x7f52cc1ed0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.798+0000 7f52d30fd700 1 --2- 172.21.15.3:0/641350918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52cc0cde60 0x7f52cc1f0e80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:54.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.798+0000 7f52d30fd700 1 -- 172.21.15.3:0/641350918 >> 172.21.15.3:0/641350918 conn(0x7f52cc070a30 msgr2=0x7f52cc152e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:54.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.798+0000 7f52d30fd700 1 -- 172.21.15.3:0/641350918 shutdown_connections 2023-03-03T06:26:54.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:54.798+0000 7f52d30fd700 1 -- 172.21.15.3:0/641350918 wait complete. 2023-03-03T06:26:54.836 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-03-03T06:26:54.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-03-03T06:26:55.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.046+0000 7ffa34850700 1 Processor -- start 2023-03-03T06:26:55.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.046+0000 7ffa34850700 1 -- start start 2023-03-03T06:26:55.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.050+0000 7ffa34850700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa300710c0 0x7ffa300cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:55.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.050+0000 7ffa34850700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa300cb9a0 0x7ffa300ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:55.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.050+0000 7ffa34850700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa3015c230 con 0x7ffa3006a780 2023-03-03T06:26:55.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.050+0000 7ffa34850700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa300caac0 con 0x7ffa300cb9a0 2023-03-03T06:26:55.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.050+0000 7ffa34850700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa300cd130 con 0x7ffa300710c0 2023-03-03T06:26:55.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.050+0000 7ffa2f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa300cb9a0 0x7ffa300ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:55.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.050+0000 7ffa2e59c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa300710c0 0x7ffa300cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:55.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.050+0000 7ffa2f59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa300cb9a0 0x7ffa300ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39124/0 (socket says 172.21.15.3:39124) 2023-03-03T06:26:55.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.050+0000 7ffa2f59e700 1 -- 172.21.15.3:0/1974466940 learned_addr learned my addr 172.21.15.3:0/1974466940 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:55.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa2f59e700 1 -- 172.21.15.3:0/1974466940 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa300710c0 msgr2=0x7ffa300cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:55.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa2f59e700 1 --2- 172.21.15.3:0/1974466940 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa300710c0 0x7ffa300cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:55.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa2f59e700 1 -- 172.21.15.3:0/1974466940 >> v1:172.21.15.32:6789/0 conn(0x7ffa3006a780 legacy=0x7ffa300709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:55.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa2f59e700 1 -- 172.21.15.3:0/1974466940 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa300cbf60 con 0x7ffa300cb9a0 2023-03-03T06:26:55.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa2f59e700 1 --2- 172.21.15.3:0/1974466940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa300cb9a0 0x7ffa300ce5e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009910 tx=0x7ffa28009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae02adbdabb0f0f5 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:55.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa2dd9b700 1 -- 172.21.15.3:0/1974466940 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28011830 con 0x7ffa300cb9a0 2023-03-03T06:26:55.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa2dd9b700 1 -- 172.21.15.3:0/1974466940 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa28011990 con 0x7ffa300cb9a0 2023-03-03T06:26:55.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa2dd9b700 1 -- 172.21.15.3:0/1974466940 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28011d40 con 0x7ffa300cb9a0 2023-03-03T06:26:55.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa34850700 1 -- 172.21.15.3:0/1974466940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa300cb9a0 msgr2=0x7ffa300ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:55.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa34850700 1 --2- 172.21.15.3:0/1974466940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa300cb9a0 0x7ffa300ce5e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009910 tx=0x7ffa28009c20 comp rx=0 tx=0).stop 2023-03-03T06:26:55.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa34850700 1 -- 172.21.15.3:0/1974466940 shutdown_connections 2023-03-03T06:26:55.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa34850700 1 --2- 172.21.15.3:0/1974466940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa300cb9a0 0x7ffa300ce5e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:55.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa34850700 1 --2- 172.21.15.3:0/1974466940 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa300710c0 0x7ffa300cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:55.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa34850700 1 -- 172.21.15.3:0/1974466940 >> 172.21.15.3:0/1974466940 conn(0x7ffa300607c0 msgr2=0x7ffa30060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:55.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa34850700 1 -- 172.21.15.3:0/1974466940 shutdown_connections 2023-03-03T06:26:55.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.054+0000 7ffa34850700 1 -- 172.21.15.3:0/1974466940 wait complete. 2023-03-03T06:26:55.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.062+0000 7ffa34850700 1 Processor -- start 2023-03-03T06:26:55.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.062+0000 7ffa34850700 1 -- start start 2023-03-03T06:26:55.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.062+0000 7ffa34850700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3006a780 0x7ffa300b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:55.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.062+0000 7ffa34850700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa300710c0 0x7ffa3013e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:55.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.062+0000 7ffa34850700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa300cb9a0 0x7ffa30156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:55.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.062+0000 7ffa34850700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa301453b0 con 0x7ffa300710c0 2023-03-03T06:26:55.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.062+0000 7ffa34850700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa301f6310 con 0x7ffa3006a780 2023-03-03T06:26:55.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.062+0000 7ffa34850700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa301f6030 con 0x7ffa300cb9a0 2023-03-03T06:26:55.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.062+0000 7ffa2e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa300710c0 0x7ffa3013e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:55.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.062+0000 7ffa2ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3006a780 0x7ffa300b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:55.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa2ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3006a780 0x7ffa300b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:39130/0 (socket says 172.21.15.3:39130) 2023-03-03T06:26:55.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa2f59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa300cb9a0 0x7ffa30156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:55.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa2ed9d700 1 -- 172.21.15.3:0/2134492924 learned_addr learned my addr 172.21.15.3:0/2134492924 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:55.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa2f59e700 1 -- 172.21.15.3:0/2134492924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3006a780 msgr2=0x7ffa300b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:55.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa2f59e700 1 --2- 172.21.15.3:0/2134492924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3006a780 0x7ffa300b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:55.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa2f59e700 1 -- 172.21.15.3:0/2134492924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa300710c0 msgr2=0x7ffa3013e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:55.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa2f59e700 1 --2- 172.21.15.3:0/2134492924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa300710c0 0x7ffa3013e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:55.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa2f59e700 1 -- 172.21.15.3:0/2134492924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa28009570 con 0x7ffa300cb9a0 2023-03-03T06:26:55.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa2f59e700 1 --2- 172.21.15.3:0/2134492924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa300cb9a0 0x7ffa30156d60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa28016000 tx=0x7ffa28004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:55.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa177fe700 1 -- 172.21.15.3:0/2134492924 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28004510 con 0x7ffa300cb9a0 2023-03-03T06:26:55.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa177fe700 1 -- 172.21.15.3:0/2134492924 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2801e070 con 0x7ffa300cb9a0 2023-03-03T06:26:55.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa177fe700 1 -- 172.21.15.3:0/2134492924 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28003900 con 0x7ffa300cb9a0 2023-03-03T06:26:55.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa34850700 1 -- 172.21.15.3:0/2134492924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa3015c7d0 con 0x7ffa300cb9a0 2023-03-03T06:26:55.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.066+0000 7ffa34850700 1 -- 172.21.15.3:0/2134492924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3015c4f0 con 0x7ffa300cb9a0 2023-03-03T06:26:55.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.070+0000 7ffa34850700 1 -- 172.21.15.3:0/2134492924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3015cab0 con 0x7ffa300cb9a0 2023-03-03T06:26:55.076 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.070+0000 7ffa177fe700 1 -- 172.21.15.3:0/2134492924 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ffa2800f050 con 0x7ffa300cb9a0 2023-03-03T06:26:55.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.074+0000 7ffa177fe700 1 --2- 172.21.15.3:0/2134492924 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ffa1c064f30 0x7ffa1c0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:55.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.074+0000 7ffa177fe700 1 -- 172.21.15.3:0/2134492924 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ffa28082c30 con 0x7ffa300cb9a0 2023-03-03T06:26:55.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.074+0000 7ffa2ed9d700 1 --2- 172.21.15.3:0/2134492924 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ffa1c064f30 0x7ffa1c0673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:55.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.074+0000 7ffa2ed9d700 1 --2- 172.21.15.3:0/2134492924 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ffa1c064f30 0x7ffa1c0673e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000f3b0 tx=0x7ffa20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:55.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.082+0000 7ffa177fe700 1 -- 172.21.15.3:0/2134492924 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ffa28050f70 con 0x7ffa300cb9a0 2023-03-03T06:26:55.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.258+0000 7ffa34850700 1 -- 172.21.15.3:0/2134492924 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ffa300b08b0 con 0x7ffa300cb9a0 2023-03-03T06:26:55.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.962+0000 7ffa177fe700 1 -- 172.21.15.3:0/2134492924 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v40) v1 ==== 138+0+0 (secure 0 0 0) 0x7ffa28054590 con 0x7ffa300cb9a0 2023-03-03T06:26:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.974+0000 7ffa34850700 1 -- 172.21.15.3:0/2134492924 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ffa1c064f30 msgr2=0x7ffa1c0673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.974+0000 7ffa34850700 1 --2- 172.21.15.3:0/2134492924 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ffa1c064f30 0x7ffa1c0673e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000f3b0 tx=0x7ffa20009040 comp rx=0 tx=0).stop 2023-03-03T06:26:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.974+0000 7ffa34850700 1 -- 172.21.15.3:0/2134492924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa300cb9a0 msgr2=0x7ffa30156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.974+0000 7ffa34850700 1 --2- 172.21.15.3:0/2134492924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa300cb9a0 0x7ffa30156d60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa28016000 tx=0x7ffa28004770 comp rx=0 tx=0).stop 2023-03-03T06:26:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.974+0000 7ffa34850700 1 -- 172.21.15.3:0/2134492924 shutdown_connections 2023-03-03T06:26:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.974+0000 7ffa34850700 1 --2- 172.21.15.3:0/2134492924 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ffa1c064f30 0x7ffa1c0673e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:55.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.974+0000 7ffa34850700 1 --2- 172.21.15.3:0/2134492924 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ffa300cb9a0 0x7ffa30156d60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:55.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.974+0000 7ffa34850700 1 --2- 172.21.15.3:0/2134492924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa300710c0 0x7ffa3013e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:55.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.974+0000 7ffa34850700 1 --2- 172.21.15.3:0/2134492924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa3006a780 0x7ffa300b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:55.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.974+0000 7ffa34850700 1 -- 172.21.15.3:0/2134492924 >> 172.21.15.3:0/2134492924 conn(0x7ffa300607c0 msgr2=0x7ffa300709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:55.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.974+0000 7ffa34850700 1 -- 172.21.15.3:0/2134492924 shutdown_connections 2023-03-03T06:26:55.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:55.974+0000 7ffa34850700 1 -- 172.21.15.3:0/2134492924 wait complete. 2023-03-03T06:26:55.982 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2023-03-03T06:26:56.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-03-03T06:26:56.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.230+0000 7f7c853cb700 1 Processor -- start 2023-03-03T06:26:56.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.230+0000 7f7c853cb700 1 -- start start 2023-03-03T06:26:56.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c853cb700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7c8015c600 0x7f7c800d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:56.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c853cb700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8006ecd0 con 0x7f7c800b5fb0 2023-03-03T06:26:56.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c853cb700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8015ffd0 con 0x7f7c8006b4f0 2023-03-03T06:26:56.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c853cb700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c800d2ed0 con 0x7f7c8015c600 2023-03-03T06:26:56.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c7ffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7c8006b4f0 0x7f7c801604b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:57928/0 (socket says 172.21.15.3:57928) 2023-03-03T06:26:56.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c7ffff700 1 -- 172.21.15.3:0/4051155764 learned_addr learned my addr 172.21.15.3:0/4051155764 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:56.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c7e7fc700 1 --1- 172.21.15.3:0/4051155764 >> v1:172.21.15.3:6789/0 conn(0x7f7c800b5fb0 0x7f7c8006e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47356/0 (socket says 172.21.15.3:47356) 2023-03-03T06:26:56.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c7effd700 1 --2- 172.21.15.3:0/4051155764 >> v2:172.21.15.3:3300/0 conn(0x7f7c8015c600 0x7f7c800d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:56.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c7dffb700 1 -- 172.21.15.3:0/4051155764 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468133073 0 0) 0x7f7c8015ffd0 con 0x7f7c8006b4f0 2023-03-03T06:26:56.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c7dffb700 1 -- 172.21.15.3:0/4051155764 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c680035d0 con 0x7f7c8006b4f0 2023-03-03T06:26:56.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c7dffb700 1 -- 172.21.15.3:0/4051155764 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486537261 0 0) 0x7f7c8006ecd0 con 0x7f7c800b5fb0 2023-03-03T06:26:56.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c7dffb700 1 -- 172.21.15.3:0/4051155764 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c8015ffd0 con 0x7f7c800b5fb0 2023-03-03T06:26:56.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c7effd700 1 -- 172.21.15.3:0/4051155764 >> v1:172.21.15.32:6789/0 conn(0x7f7c8006b4f0 legacy=0x7f7c801604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:56.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c7effd700 1 -- 172.21.15.3:0/4051155764 >> v1:172.21.15.3:6789/0 conn(0x7f7c800b5fb0 legacy=0x7f7c8006e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:56.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.234+0000 7f7c7effd700 1 -- 172.21.15.3:0/4051155764 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c800b6470 con 0x7f7c8015c600 2023-03-03T06:26:56.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.238+0000 7f7c7effd700 1 --2- 172.21.15.3:0/4051155764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c8015c600 0x7f7c800d0700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009850 tx=0x7f7c70009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323f50f7ecbd0687 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:56.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.238+0000 7f7c7dffb700 1 -- 172.21.15.3:0/4051155764 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70011830 con 0x7f7c8015c600 2023-03-03T06:26:56.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.238+0000 7f7c7dffb700 1 -- 172.21.15.3:0/4051155764 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c70011990 con 0x7f7c8015c600 2023-03-03T06:26:56.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.238+0000 7f7c7dffb700 1 -- 172.21.15.3:0/4051155764 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70011d40 con 0x7f7c8015c600 2023-03-03T06:26:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.238+0000 7f7c853cb700 1 -- 172.21.15.3:0/4051155764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c8015c600 msgr2=0x7f7c800d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.238+0000 7f7c853cb700 1 --2- 172.21.15.3:0/4051155764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c8015c600 0x7f7c800d0700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009850 tx=0x7f7c70009b60 comp rx=0 tx=0).stop 2023-03-03T06:26:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.238+0000 7f7c853cb700 1 -- 172.21.15.3:0/4051155764 shutdown_connections 2023-03-03T06:26:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.238+0000 7f7c853cb700 1 --2- 172.21.15.3:0/4051155764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c8015c600 0x7f7c800d0700 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.238+0000 7f7c853cb700 1 -- 172.21.15.3:0/4051155764 >> 172.21.15.3:0/4051155764 conn(0x7f7c800d4530 msgr2=0x7f7c800d4930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.238+0000 7f7c853cb700 1 -- 172.21.15.3:0/4051155764 shutdown_connections 2023-03-03T06:26:56.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.238+0000 7f7c853cb700 1 -- 172.21.15.3:0/4051155764 wait complete. 2023-03-03T06:26:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c853cb700 1 Processor -- start 2023-03-03T06:26:56.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c853cb700 1 -- start start 2023-03-03T06:26:56.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c853cb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c8006b4f0 0x7f7c800cff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:56.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c853cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c800b5fb0 0x7f7c801f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:56.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c853cb700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c8015c600 0x7f7c800cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:56.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c853cb700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800cfae0 con 0x7f7c8006b4f0 2023-03-03T06:26:56.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c853cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800cf1f0 con 0x7f7c800b5fb0 2023-03-03T06:26:56.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c853cb700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800ce4d0 con 0x7f7c8015c600 2023-03-03T06:26:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c7effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c8006b4f0 0x7f7c800cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:56.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c7effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c8006b4f0 0x7f7c800cff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52296/0 (socket says 172.21.15.3:52296) 2023-03-03T06:26:56.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c7effd700 1 -- 172.21.15.3:0/2251409185 learned_addr learned my addr 172.21.15.3:0/2251409185 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:56.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c7e7fc700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c800b5fb0 0x7f7c801f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:56.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c7ffff700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c8015c600 0x7f7c800cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:56.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c7effd700 1 -- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c8015c600 msgr2=0x7f7c800cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:56.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c7effd700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c8015c600 0x7f7c800cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:56.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c7effd700 1 -- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c800b5fb0 msgr2=0x7f7c801f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:56.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c7effd700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c800b5fb0 0x7f7c801f2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c7effd700 1 -- 172.21.15.3:0/2251409185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c700094b0 con 0x7f7c8006b4f0 2023-03-03T06:26:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.242+0000 7f7c7e7fc700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c800b5fb0 0x7f7c801f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:26:56.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.246+0000 7f7c7effd700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c8006b4f0 0x7f7c800cff70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7c70014040 tx=0x7f7c70004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.246+0000 7f7c677fe700 1 -- 172.21.15.3:0/2251409185 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70004590 con 0x7f7c8006b4f0 2023-03-03T06:26:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.246+0000 7f7c677fe700 1 -- 172.21.15.3:0/2251409185 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7001e070 con 0x7f7c8006b4f0 2023-03-03T06:26:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.246+0000 7f7c677fe700 1 -- 172.21.15.3:0/2251409185 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70003900 con 0x7f7c8006b4f0 2023-03-03T06:26:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.246+0000 7f7c853cb700 1 -- 172.21.15.3:0/2251409185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c800b6a30 con 0x7f7c8006b4f0 2023-03-03T06:26:56.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.246+0000 7f7c853cb700 1 -- 172.21.15.3:0/2251409185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c800b6750 con 0x7f7c8006b4f0 2023-03-03T06:26:56.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.246+0000 7f7c853cb700 1 -- 172.21.15.3:0/2251409185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c800b6d10 con 0x7f7c8006b4f0 2023-03-03T06:26:56.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.250+0000 7f7c677fe700 1 -- 172.21.15.3:0/2251409185 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7c7002a020 con 0x7f7c8006b4f0 2023-03-03T06:26:56.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.254+0000 7f7c677fe700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7c6c064ee0 0x7f7c6c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:56.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.254+0000 7f7c677fe700 1 -- 172.21.15.3:0/2251409185 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f7c70083130 con 0x7f7c8006b4f0 2023-03-03T06:26:56.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.254+0000 7f7c7e7fc700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7c6c064ee0 0x7f7c6c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:56.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.254+0000 7f7c7e7fc700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7c6c064ee0 0x7f7c6c067390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7c74007fd0 tx=0x7f7c74007c40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:56.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.258+0000 7f7c677fe700 1 -- 172.21.15.3:0/2251409185 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7c70051300 con 0x7f7c8006b4f0 2023-03-03T06:26:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.438+0000 7f7c853cb700 1 -- 172.21.15.3:0/2251409185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7c8004e180 con 0x7f7c8006b4f0 2023-03-03T06:26:56.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.978+0000 7f7c677fe700 1 -- 172.21.15.3:0/2251409185 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v41) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7c70054920 con 0x7f7c8006b4f0 2023-03-03T06:26:57.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.998+0000 7f7c853cb700 1 -- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7c6c064ee0 msgr2=0x7f7c6c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:56.998+0000 7f7c853cb700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7c6c064ee0 0x7f7c6c067390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7c74007fd0 tx=0x7f7c74007c40 comp rx=0 tx=0).stop 2023-03-03T06:26:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.002+0000 7f7c853cb700 1 -- 172.21.15.3:0/2251409185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c8006b4f0 msgr2=0x7f7c800cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.002+0000 7f7c853cb700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c8006b4f0 0x7f7c800cff70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7c70014040 tx=0x7f7c70004770 comp rx=0 tx=0).stop 2023-03-03T06:26:57.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.002+0000 7f7c853cb700 1 -- 172.21.15.3:0/2251409185 shutdown_connections 2023-03-03T06:26:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.002+0000 7f7c853cb700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7c6c064ee0 0x7f7c6c067390 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.002+0000 7f7c853cb700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f7c8015c600 0x7f7c800cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.002+0000 7f7c853cb700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c800b5fb0 0x7f7c801f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.002+0000 7f7c853cb700 1 --2- 172.21.15.3:0/2251409185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c8006b4f0 0x7f7c800cff70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.002+0000 7f7c853cb700 1 -- 172.21.15.3:0/2251409185 >> 172.21.15.3:0/2251409185 conn(0x7f7c800d4530 msgr2=0x7f7c8006e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:57.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.002+0000 7f7c853cb700 1 -- 172.21.15.3:0/2251409185 shutdown_connections 2023-03-03T06:26:57.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.002+0000 7f7c853cb700 1 -- 172.21.15.3:0/2251409185 wait complete. 2023-03-03T06:26:57.015 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2023-03-03T06:26:57.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-03-03T06:26:57.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.250+0000 7fc34f1d2700 1 Processor -- start 2023-03-03T06:26:57.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.250+0000 7fc34f1d2700 1 -- start start 2023-03-03T06:26:57.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34f1d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc34815d720 0x7fc3481608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:57.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34f1d2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3481600f0 con 0x7fc3480cf610 2023-03-03T06:26:57.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34f1d2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3480cd420 con 0x7fc348156cc0 2023-03-03T06:26:57.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34f1d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480cd700 con 0x7fc34815d720 2023-03-03T06:26:57.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34e9d1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc3480cf610 0x7fc3480d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47360/0 (socket says 172.21.15.3:47360) 2023-03-03T06:26:57.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34e9d1700 1 -- 172.21.15.3:0/1485223237 learned_addr learned my addr 172.21.15.3:0/1485223237 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:57.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34d9cf700 1 --2- 172.21.15.3:0/1485223237 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc34815d720 0x7fc3481608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:57.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34d1ce700 1 -- 172.21.15.3:0/1485223237 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128058253 0 0) 0x7fc3481600f0 con 0x7fc3480cf610 2023-03-03T06:26:57.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34d1ce700 1 -- 172.21.15.3:0/1485223237 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3340035d0 con 0x7fc3480cf610 2023-03-03T06:26:57.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34d1ce700 1 -- 172.21.15.3:0/1485223237 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906738981 0 0) 0x7fc3480cd420 con 0x7fc348156cc0 2023-03-03T06:26:57.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34d1ce700 1 -- 172.21.15.3:0/1485223237 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3481600f0 con 0x7fc348156cc0 2023-03-03T06:26:57.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34d9cf700 1 -- 172.21.15.3:0/1485223237 >> v1:172.21.15.32:6789/0 conn(0x7fc348156cc0 legacy=0x7fc3480713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:57.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34d9cf700 1 -- 172.21.15.3:0/1485223237 >> v1:172.21.15.3:6789/0 conn(0x7fc3480cf610 legacy=0x7fc3480d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:57.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.254+0000 7fc34d9cf700 1 -- 172.21.15.3:0/1485223237 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc348160390 con 0x7fc34815d720 2023-03-03T06:26:57.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.258+0000 7fc34d9cf700 1 --2- 172.21.15.3:0/1485223237 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc34815d720 0x7fc3481608b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc338009880 tx=0x7fc338009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=708ec4bd97f392fa server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:57.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.258+0000 7fc34d1ce700 1 -- 172.21.15.3:0/1485223237 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338011830 con 0x7fc34815d720 2023-03-03T06:26:57.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.258+0000 7fc34d1ce700 1 -- 172.21.15.3:0/1485223237 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc338011990 con 0x7fc34815d720 2023-03-03T06:26:57.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.258+0000 7fc34d1ce700 1 -- 172.21.15.3:0/1485223237 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338011d20 con 0x7fc34815d720 2023-03-03T06:26:57.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.258+0000 7fc34f1d2700 1 -- 172.21.15.3:0/1485223237 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc34815d720 msgr2=0x7fc3481608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:57.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.258+0000 7fc34f1d2700 1 --2- 172.21.15.3:0/1485223237 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc34815d720 0x7fc3481608b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc338009880 tx=0x7fc338009b90 comp rx=0 tx=0).stop 2023-03-03T06:26:57.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.258+0000 7fc34f1d2700 1 -- 172.21.15.3:0/1485223237 shutdown_connections 2023-03-03T06:26:57.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.258+0000 7fc34f1d2700 1 --2- 172.21.15.3:0/1485223237 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc34815d720 0x7fc3481608b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.258+0000 7fc34f1d2700 1 -- 172.21.15.3:0/1485223237 >> 172.21.15.3:0/1485223237 conn(0x7fc34806e5c0 msgr2=0x7fc34806e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:57.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.258+0000 7fc34f1d2700 1 -- 172.21.15.3:0/1485223237 shutdown_connections 2023-03-03T06:26:57.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.258+0000 7fc34f1d2700 1 -- 172.21.15.3:0/1485223237 wait complete. 2023-03-03T06:26:57.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.262+0000 7fc34f1d2700 1 Processor -- start 2023-03-03T06:26:57.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.262+0000 7fc34f1d2700 1 -- start start 2023-03-03T06:26:57.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.262+0000 7fc34f1d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3480cf610 0x7fc348154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:57.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.262+0000 7fc34f1d2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc348156cc0 0x7fc3480c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:57.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.262+0000 7fc34f1d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc34815d720 0x7fc3480d1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:57.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.262+0000 7fc34f1d2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc348154b70 con 0x7fc348156cc0 2023-03-03T06:26:57.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.262+0000 7fc34f1d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480c92d0 con 0x7fc3480cf610 2023-03-03T06:26:57.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.262+0000 7fc34f1d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480caf20 con 0x7fc34815d720 2023-03-03T06:26:57.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.262+0000 7fc34d9cf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc348156cc0 0x7fc3480c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:57.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.262+0000 7fc34d9cf700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc348156cc0 0x7fc3480c5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52312/0 (socket says 172.21.15.3:52312) 2023-03-03T06:26:57.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.262+0000 7fc34d9cf700 1 -- 172.21.15.3:0/2032508853 learned_addr learned my addr 172.21.15.3:0/2032508853 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:57.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc34e9d1700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc34815d720 0x7fc3480d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:57.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc34e1d0700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3480cf610 0x7fc348154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:57.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc34d9cf700 1 -- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc34815d720 msgr2=0x7fc3480d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:57.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc34d9cf700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc34815d720 0x7fc3480d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc34d9cf700 1 -- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3480cf610 msgr2=0x7fc348154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:57.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc34d9cf700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3480cf610 0x7fc348154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc34d9cf700 1 -- 172.21.15.3:0/2032508853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380094b0 con 0x7fc348156cc0 2023-03-03T06:26:57.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc34e9d1700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc34815d720 0x7fc3480d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:57.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc34e1d0700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3480cf610 0x7fc348154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:26:57.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc34d9cf700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc348156cc0 0x7fc3480c5b20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc3380090b0 tx=0x7fc338004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:57.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc33effd700 1 -- 172.21.15.3:0/2032508853 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33801a8c0 con 0x7fc348156cc0 2023-03-03T06:26:57.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc33effd700 1 -- 172.21.15.3:0/2032508853 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33801e070 con 0x7fc348156cc0 2023-03-03T06:26:57.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc33effd700 1 -- 172.21.15.3:0/2032508853 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33801ac40 con 0x7fc348156cc0 2023-03-03T06:26:57.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc34f1d2700 1 -- 172.21.15.3:0/2032508853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc348160df0 con 0x7fc348156cc0 2023-03-03T06:26:57.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc34f1d2700 1 -- 172.21.15.3:0/2032508853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3480cd9e0 con 0x7fc348156cc0 2023-03-03T06:26:57.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.266+0000 7fc34f1d2700 1 -- 172.21.15.3:0/2032508853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc348160f80 con 0x7fc348156cc0 2023-03-03T06:26:57.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.270+0000 7fc33effd700 1 -- 172.21.15.3:0/2032508853 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc33800f050 con 0x7fc348156cc0 2023-03-03T06:26:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.274+0000 7fc33effd700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc32c064f10 0x7fc32c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.274+0000 7fc33effd700 1 -- 172.21.15.3:0/2032508853 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fc338082bc0 con 0x7fc348156cc0 2023-03-03T06:26:57.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.274+0000 7fc34e1d0700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc32c064f10 0x7fc32c0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:57.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.274+0000 7fc34e1d0700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc32c064f10 0x7fc32c0673c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc3440011e0 tx=0x7fc344009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:57.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.282+0000 7fc33effd700 1 -- 172.21.15.3:0/2032508853 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc338050b90 con 0x7fc348156cc0 2023-03-03T06:26:57.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:26:57.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:26:57.500 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.494+0000 7fc34f1d2700 1 -- 172.21.15.3:0/2032508853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc3480cad80 con 0x7fc348156cc0 2023-03-03T06:26:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.510+0000 7fc33effd700 1 -- 172.21.15.3:0/2032508853 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc3480cad80 con 0x7fc348156cc0 2023-03-03T06:26:57.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.518+0000 7fc34f1d2700 1 -- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc32c064f10 msgr2=0x7fc32c0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:57.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.518+0000 7fc34f1d2700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc32c064f10 0x7fc32c0673c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc3440011e0 tx=0x7fc344009040 comp rx=0 tx=0).stop 2023-03-03T06:26:57.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.518+0000 7fc34f1d2700 1 -- 172.21.15.3:0/2032508853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc348156cc0 msgr2=0x7fc3480c5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:57.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.518+0000 7fc34f1d2700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc348156cc0 0x7fc3480c5b20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc3380090b0 tx=0x7fc338004770 comp rx=0 tx=0).stop 2023-03-03T06:26:57.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.518+0000 7fc34f1d2700 1 -- 172.21.15.3:0/2032508853 shutdown_connections 2023-03-03T06:26:57.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.518+0000 7fc34f1d2700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc32c064f10 0x7fc32c0673c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.518+0000 7fc34f1d2700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc34815d720 0x7fc3480d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.518+0000 7fc34f1d2700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc348156cc0 0x7fc3480c5b20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.518+0000 7fc34f1d2700 1 --2- 172.21.15.3:0/2032508853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3480cf610 0x7fc348154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.518+0000 7fc34f1d2700 1 -- 172.21.15.3:0/2032508853 >> 172.21.15.3:0/2032508853 conn(0x7fc34806e5c0 msgr2=0x7fc3480b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:57.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.522+0000 7fc34f1d2700 1 -- 172.21.15.3:0/2032508853 shutdown_connections 2023-03-03T06:26:57.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.522+0000 7fc34f1d2700 1 -- 172.21.15.3:0/2032508853 wait complete. 2023-03-03T06:26:57.550 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 6 and data pool 7 2023-03-03T06:26:57.577 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:57.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.834+0000 7f217d7f9700 1 Processor -- start 2023-03-03T06:26:57.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.834+0000 7f217d7f9700 1 -- start start 2023-03-03T06:26:57.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.838+0000 7f217d7f9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21781608d0 0x7f21780cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:57.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.838+0000 7f217d7f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21780cfb60 0x7f21780d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:57.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.838+0000 7f217d7f9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2178160440 con 0x7f2178153670 2023-03-03T06:26:57.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.838+0000 7f217d7f9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21780cffe0 con 0x7f21781608d0 2023-03-03T06:26:57.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.838+0000 7f217d7f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21780cd3e0 con 0x7f21780cfb60 2023-03-03T06:26:57.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.838+0000 7f2177fff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2178153670 0x7f2178071350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:57944/0 (socket says 172.21.15.3:57944) 2023-03-03T06:26:57.844 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.838+0000 7f2177fff700 1 -- 172.21.15.3:0/2569032128 learned_addr learned my addr 172.21.15.3:0/2569032128 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.838+0000 7f216f7fe700 1 --2- 172.21.15.3:0/2569032128 >> v2:172.21.15.32:3300/0 conn(0x7f21781608d0 0x7f21780cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.838+0000 7f217cff8700 1 --2- 172.21.15.3:0/2569032128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21780cfb60 0x7f21780d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:57.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.838+0000 7f21777fe700 1 -- 172.21.15.3:0/2569032128 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849823812 0 0) 0x7f2178160440 con 0x7f2178153670 2023-03-03T06:26:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f21777fe700 1 -- 172.21.15.3:0/2569032128 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21640035d0 con 0x7f2178153670 2023-03-03T06:26:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f217cff8700 1 -- 172.21.15.3:0/2569032128 >> v2:172.21.15.32:3300/0 conn(0x7f21781608d0 msgr2=0x7f21780cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f217cff8700 1 --2- 172.21.15.3:0/2569032128 >> v2:172.21.15.32:3300/0 conn(0x7f21781608d0 0x7f21780cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f217cff8700 1 -- 172.21.15.3:0/2569032128 >> v1:172.21.15.32:6789/0 conn(0x7f2178153670 legacy=0x7f2178071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:57.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f217cff8700 1 -- 172.21.15.3:0/2569032128 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21780cd6c0 con 0x7f21780cfb60 2023-03-03T06:26:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f217cff8700 1 --2- 172.21.15.3:0/2569032128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21780cfb60 0x7f21780d0b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2170009dc0 tx=0x7f217000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bbcc8bbdf4c0183 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f21777fe700 1 -- 172.21.15.3:0/2569032128 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217000e070 con 0x7f21780cfb60 2023-03-03T06:26:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f21777fe700 1 -- 172.21.15.3:0/2569032128 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217000bae0 con 0x7f21780cfb60 2023-03-03T06:26:57.847 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f21777fe700 1 -- 172.21.15.3:0/2569032128 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217000be90 con 0x7f21780cfb60 2023-03-03T06:26:57.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f217d7f9700 1 -- 172.21.15.3:0/2569032128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21780cfb60 msgr2=0x7f21780d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:57.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f217d7f9700 1 --2- 172.21.15.3:0/2569032128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21780cfb60 0x7f21780d0b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2170009dc0 tx=0x7f217000b330 comp rx=0 tx=0).stop 2023-03-03T06:26:57.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f217d7f9700 1 -- 172.21.15.3:0/2569032128 shutdown_connections 2023-03-03T06:26:57.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f217d7f9700 1 --2- 172.21.15.3:0/2569032128 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21780cfb60 0x7f21780d0b10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f217d7f9700 1 --2- 172.21.15.3:0/2569032128 >> v2:172.21.15.32:3300/0 conn(0x7f21781608d0 0x7f21780cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f217d7f9700 1 -- 172.21.15.3:0/2569032128 >> 172.21.15.3:0/2569032128 conn(0x7f217806e5c0 msgr2=0x7f217806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:57.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f217d7f9700 1 -- 172.21.15.3:0/2569032128 shutdown_connections 2023-03-03T06:26:57.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.842+0000 7f217d7f9700 1 -- 172.21.15.3:0/2569032128 wait complete. 2023-03-03T06:26:57.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.846+0000 7f217d7f9700 1 Processor -- start 2023-03-03T06:26:57.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.846+0000 7f217d7f9700 1 -- start start 2023-03-03T06:26:57.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.846+0000 7f217d7f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21780cfb60 0x7f21780b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:57.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.846+0000 7f217d7f9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2178153670 0x7f21780b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:57.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f217d7f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21781608d0 0x7f21780b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:57.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f217d7f9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21780cb870 con 0x7f2178153670 2023-03-03T06:26:57.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f217d7f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21780cbac0 con 0x7f21781608d0 2023-03-03T06:26:57.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f217d7f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21780cbda0 con 0x7f21780cfb60 2023-03-03T06:26:57.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f216f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2178153670 0x7f21780b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:57.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f216f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2178153670 0x7f21780b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52318/0 (socket says 172.21.15.3:52318) 2023-03-03T06:26:57.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f216f7fe700 1 -- 172.21.15.3:0/586692349 learned_addr learned my addr 172.21.15.3:0/586692349 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:57.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f2177fff700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21780cfb60 0x7f21780b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:57.855 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f217cff8700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21781608d0 0x7f21780b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:57.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f2177fff700 1 -- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21781608d0 msgr2=0x7f21780b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:57.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f2177fff700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21781608d0 0x7f21780b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.856 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f2177fff700 1 -- 172.21.15.3:0/586692349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2178153670 msgr2=0x7f21780b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:57.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f2177fff700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2178153670 0x7f21780b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:57.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f2177fff700 1 -- 172.21.15.3:0/586692349 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2170009a70 con 0x7f21780cfb60 2023-03-03T06:26:57.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f217cff8700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21781608d0 0x7f21780b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:57.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f216f7fe700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2178153670 0x7f21780b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:26:57.857 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f2177fff700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21780cfb60 0x7f21780b4cc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f216800c500 tx=0x7f216800c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:57.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f21757fa700 1 -- 172.21.15.3:0/586692349 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168005d90 con 0x7f21780cfb60 2023-03-03T06:26:57.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f21757fa700 1 -- 172.21.15.3:0/586692349 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21680070d0 con 0x7f21780cfb60 2023-03-03T06:26:57.858 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f21757fa700 1 -- 172.21.15.3:0/586692349 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168007440 con 0x7f21780cfb60 2023-03-03T06:26:57.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f217d7f9700 1 -- 172.21.15.3:0/586692349 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21780cd9a0 con 0x7f21780cfb60 2023-03-03T06:26:57.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.850+0000 7f217d7f9700 1 -- 172.21.15.3:0/586692349 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21781600d0 con 0x7f21780cfb60 2023-03-03T06:26:57.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.854+0000 7f217d7f9700 1 -- 172.21.15.3:0/586692349 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21780cdb30 con 0x7f21780cfb60 2023-03-03T06:26:57.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.854+0000 7f21757fa700 1 -- 172.21.15.3:0/586692349 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f2168019030 con 0x7f21780cfb60 2023-03-03T06:26:57.863 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.858+0000 7f21757fa700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2158064fb0 0x7f2158067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:57.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.858+0000 7f21757fa700 1 -- 172.21.15.3:0/586692349 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2168081240 con 0x7f21780cfb60 2023-03-03T06:26:57.864 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.858+0000 7f216f7fe700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2158064fb0 0x7f2158067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:57.865 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.862+0000 7f216f7fe700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2158064fb0 0x7f2158067460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f21780b6580 tx=0x7f2160009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:57.873 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:57.866+0000 7f21757fa700 1 -- 172.21.15.3:0/586692349 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f216804a400 con 0x7f21780cfb60 2023-03-03T06:26:58.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.050+0000 7f217d7f9700 1 -- 172.21.15.3:0/586692349 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21780b68f0 con 0x7f21780cfb60 2023-03-03T06:26:58.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.054+0000 7f21757fa700 1 -- 172.21.15.3:0/586692349 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17949 (secure 0 0 0) 0x7f216804f2e0 con 0x7f21780cfb60 2023-03-03T06:26:58.058 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:58.059 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":42,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:26:57.499502+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:26:55.353616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-03-03T06:26:56.534268+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:58.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.058+0000 7f217d7f9700 1 -- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2158064fb0 msgr2=0x7f2158067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:58.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.058+0000 7f217d7f9700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2158064fb0 0x7f2158067460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f21780b6580 tx=0x7f2160009240 comp rx=0 tx=0).stop 2023-03-03T06:26:58.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.058+0000 7f217d7f9700 1 -- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21780cfb60 msgr2=0x7f21780b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:58.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.058+0000 7f217d7f9700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21780cfb60 0x7f21780b4cc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f216800c500 tx=0x7f216800c810 comp rx=0 tx=0).stop 2023-03-03T06:26:58.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.058+0000 7f217d7f9700 1 -- 172.21.15.3:0/586692349 shutdown_connections 2023-03-03T06:26:58.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.058+0000 7f217d7f9700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2158064fb0 0x7f2158067460 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.058+0000 7f217d7f9700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21781608d0 0x7f21780b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.058+0000 7f217d7f9700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2178153670 0x7f21780b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.058+0000 7f217d7f9700 1 --2- 172.21.15.3:0/586692349 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f21780cfb60 0x7f21780b4cc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.058+0000 7f217d7f9700 1 -- 172.21.15.3:0/586692349 >> 172.21.15.3:0/586692349 conn(0x7f217806e5c0 msgr2=0x7f217815ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.058+0000 7f217d7f9700 1 -- 172.21.15.3:0/586692349 shutdown_connections 2023-03-03T06:26:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.058+0000 7f217d7f9700 1 -- 172.21.15.3:0/586692349 wait complete. 2023-03-03T06:26:58.098 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:26:58.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.298+0000 7f328db43700 1 Processor -- start 2023-03-03T06:26:58.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.298+0000 7f328db43700 1 -- start start 2023-03-03T06:26:58.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.302+0000 7f328db43700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3288158440 0x7f32880c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:58.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.302+0000 7f328db43700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f328815c750 0x7f32880ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:58.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.302+0000 7f328db43700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32880cdd30 con 0x7f32880aee70 2023-03-03T06:26:58.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.302+0000 7f328db43700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32880ca4d0 con 0x7f328815c750 2023-03-03T06:26:58.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.302+0000 7f328db43700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32880cd6e0 con 0x7f3288158440 2023-03-03T06:26:58.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.302+0000 7f327ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f32880aee70 0x7f328815c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47366/0 (socket says 172.21.15.3:47366) 2023-03-03T06:26:58.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.302+0000 7f327ffff700 1 -- 172.21.15.3:0/3909182518 learned_addr learned my addr 172.21.15.3:0/3909182518 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:26:58.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.302+0000 7f328cb41700 1 --2- 172.21.15.3:0/3909182518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3288158440 0x7f32880c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:58.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.302+0000 7f328d342700 1 --2- 172.21.15.3:0/3909182518 >> v2:172.21.15.32:3300/0 conn(0x7f328815c750 0x7f32880ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:58.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.302+0000 7f327f7fe700 1 -- 172.21.15.3:0/3909182518 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1681503525 0 0) 0x7f32880cdd30 con 0x7f32880aee70 2023-03-03T06:26:58.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.302+0000 7f327f7fe700 1 -- 172.21.15.3:0/3909182518 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32700035d0 con 0x7f32880aee70 2023-03-03T06:26:58.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f328d342700 1 -- 172.21.15.3:0/3909182518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3288158440 msgr2=0x7f32880c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:58.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f328d342700 1 --2- 172.21.15.3:0/3909182518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3288158440 0x7f32880c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f328d342700 1 -- 172.21.15.3:0/3909182518 >> v1:172.21.15.3:6789/0 conn(0x7f32880aee70 legacy=0x7f328815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:58.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f328d342700 1 -- 172.21.15.3:0/3909182518 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32880cd9c0 con 0x7f328815c750 2023-03-03T06:26:58.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f328d342700 1 --2- 172.21.15.3:0/3909182518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f328815c750 0x7f32880ce1c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f328000aea0 tx=0x7f328000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1edf3a27a749a8e2 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:58.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f327f7fe700 1 -- 172.21.15.3:0/3909182518 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328000f070 con 0x7f328815c750 2023-03-03T06:26:58.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f327f7fe700 1 -- 172.21.15.3:0/3909182518 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328000a2e0 con 0x7f328815c750 2023-03-03T06:26:58.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f327f7fe700 1 -- 172.21.15.3:0/3909182518 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328000cb50 con 0x7f328815c750 2023-03-03T06:26:58.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f328db43700 1 -- 172.21.15.3:0/3909182518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f328815c750 msgr2=0x7f32880ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:58.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f328db43700 1 --2- 172.21.15.3:0/3909182518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f328815c750 0x7f32880ce1c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f328000aea0 tx=0x7f328000c330 comp rx=0 tx=0).stop 2023-03-03T06:26:58.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f328db43700 1 -- 172.21.15.3:0/3909182518 shutdown_connections 2023-03-03T06:26:58.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f328db43700 1 --2- 172.21.15.3:0/3909182518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f328815c750 0x7f32880ce1c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f328db43700 1 --2- 172.21.15.3:0/3909182518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3288158440 0x7f32880c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f328db43700 1 -- 172.21.15.3:0/3909182518 >> 172.21.15.3:0/3909182518 conn(0x7f3288070a30 msgr2=0x7f3288070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:58.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f328db43700 1 -- 172.21.15.3:0/3909182518 shutdown_connections 2023-03-03T06:26:58.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.306+0000 7f328db43700 1 -- 172.21.15.3:0/3909182518 wait complete. 2023-03-03T06:26:58.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.310+0000 7f328db43700 1 Processor -- start 2023-03-03T06:26:58.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.310+0000 7f328db43700 1 -- start start 2023-03-03T06:26:58.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f328db43700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32880aee70 0x7f328814ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:58.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f328db43700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3288158440 0x7f328815a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:58.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f328db43700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f328815c750 0x7f32880cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:58.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f328db43700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3288159ce0 con 0x7f3288158440 2023-03-03T06:26:58.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f328db43700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328814b0b0 con 0x7f32880aee70 2023-03-03T06:26:58.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f328db43700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32881590d0 con 0x7f328815c750 2023-03-03T06:26:58.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f327ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3288158440 0x7f328815a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:58.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f327ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3288158440 0x7f328815a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52328/0 (socket says 172.21.15.3:52328) 2023-03-03T06:26:58.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f327ffff700 1 -- 172.21.15.3:0/1596319135 learned_addr learned my addr 172.21.15.3:0/1596319135 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:58.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f328cb41700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32880aee70 0x7f328814ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:58.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f328d342700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f328815c750 0x7f32880cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:58.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f327ffff700 1 -- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f328815c750 msgr2=0x7f32880cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:58.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f327ffff700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f328815c750 0x7f32880cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f327ffff700 1 -- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32880aee70 msgr2=0x7f328814ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:58.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f327ffff700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32880aee70 0x7f328814ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f327ffff700 1 -- 172.21.15.3:0/1596319135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328000ab80 con 0x7f3288158440 2023-03-03T06:26:58.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f327ffff700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3288158440 0x7f328815a590 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3278005930 tx=0x7f327800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:58.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f327d7fa700 1 -- 172.21.15.3:0/1596319135 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278005d90 con 0x7f3288158440 2023-03-03T06:26:58.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f327d7fa700 1 -- 172.21.15.3:0/1596319135 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32780070d0 con 0x7f3288158440 2023-03-03T06:26:58.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f328db43700 1 -- 172.21.15.3:0/1596319135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328815cd90 con 0x7f3288158440 2023-03-03T06:26:58.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f327d7fa700 1 -- 172.21.15.3:0/1596319135 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278007480 con 0x7f3288158440 2023-03-03T06:26:58.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.314+0000 7f328db43700 1 -- 172.21.15.3:0/1596319135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32880ca7b0 con 0x7f3288158440 2023-03-03T06:26:58.324 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.318+0000 7f328db43700 1 -- 172.21.15.3:0/1596319135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3288158810 con 0x7f3288158440 2023-03-03T06:26:58.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.318+0000 7f327d7fa700 1 -- 172.21.15.3:0/1596319135 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3278019030 con 0x7f3288158440 2023-03-03T06:26:58.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.322+0000 7f327d7fa700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3274064f10 0x7f32740673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:58.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.322+0000 7f327d7fa700 1 -- 172.21.15.3:0/1596319135 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3278081360 con 0x7f3288158440 2023-03-03T06:26:58.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.322+0000 7f328cb41700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3274064f10 0x7f32740673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:58.329 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.326+0000 7f328cb41700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3274064f10 0x7f32740673c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3284009bf0 tx=0x7f3284009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:58.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.326+0000 7f327d7fa700 1 -- 172.21.15.3:0/1596319135 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f327804f360 con 0x7f3288158440 2023-03-03T06:26:58.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.502+0000 7f328db43700 1 -- 172.21.15.3:0/1596319135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3288073e90 con 0x7f3288158440 2023-03-03T06:26:58.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.506+0000 7f327d7fa700 1 -- 172.21.15.3:0/1596319135 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17949 (secure 0 0 0) 0x7f3278052980 con 0x7f3288158440 2023-03-03T06:26:58.512 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:26:58.513 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":42,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:26:57.499502+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:26:55.353616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-03-03T06:26:56.534268+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:26:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.514+0000 7f328db43700 1 -- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3274064f10 msgr2=0x7f32740673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:58.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.514+0000 7f328db43700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3274064f10 0x7f32740673c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3284009bf0 tx=0x7f3284009240 comp rx=0 tx=0).stop 2023-03-03T06:26:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.514+0000 7f328db43700 1 -- 172.21.15.3:0/1596319135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3288158440 msgr2=0x7f328815a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.514+0000 7f328db43700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3288158440 0x7f328815a590 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3278005930 tx=0x7f327800c9e0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.514+0000 7f328db43700 1 -- 172.21.15.3:0/1596319135 shutdown_connections 2023-03-03T06:26:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.514+0000 7f328db43700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3274064f10 0x7f32740673c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.514+0000 7f328db43700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f328815c750 0x7f32880cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.514+0000 7f328db43700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3288158440 0x7f328815a590 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.514+0000 7f328db43700 1 --2- 172.21.15.3:0/1596319135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32880aee70 0x7f328814ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.514+0000 7f328db43700 1 -- 172.21.15.3:0/1596319135 >> 172.21.15.3:0/1596319135 conn(0x7f3288070a30 msgr2=0x7f3288152b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.514+0000 7f328db43700 1 -- 172.21.15.3:0/1596319135 shutdown_connections 2023-03-03T06:26:58.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.514+0000 7f328db43700 1 -- 172.21.15.3:0/1596319135 wait complete. 2023-03-03T06:26:58.553 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-03-03T06:26:58.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.794+0000 7f1ea3985700 1 Processor -- start 2023-03-03T06:26:58.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.794+0000 7f1ea3985700 1 -- start start 2023-03-03T06:26:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.798+0000 7f1ea3985700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e9c158400 0x7f1e9c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:58.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.798+0000 7f1ea3985700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e9c0cb9a0 0x7f1e9c0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:58.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.798+0000 7f1ea3985700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1e9c0709b0 0x7f1e9c070e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:58.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.798+0000 7f1ea3985700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e9c0caac0 con 0x7f1e9c0709b0 2023-03-03T06:26:58.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.798+0000 7f1ea3985700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e9c071370 con 0x7f1e9c0cb9a0 2023-03-03T06:26:58.805 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.798+0000 7f1ea3985700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c15c230 con 0x7f1e9c158400 2023-03-03T06:26:58.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.798+0000 7f1ea1f22700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1e9c0709b0 0x7f1e9c070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:58.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.798+0000 7f1ea1f22700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1e9c0709b0 0x7f1e9c070e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52344/0 (socket says 172.21.15.3:52344) 2023-03-03T06:26:58.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.798+0000 7f1ea0f20700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e9c0cb9a0 0x7f1e9c0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:58.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.798+0000 7f1ea1f22700 1 -- 172.21.15.3:0/3777915974 learned_addr learned my addr 172.21.15.3:0/3777915974 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:58.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.798+0000 7f1ea1721700 1 --2- 172.21.15.3:0/3777915974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e9c158400 0x7f1e9c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:58.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1ea1f22700 1 -- 172.21.15.3:0/3777915974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e9c158400 msgr2=0x7f1e9c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:58.807 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1ea1f22700 1 --2- 172.21.15.3:0/3777915974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e9c158400 0x7f1e9c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1ea1f22700 1 -- 172.21.15.3:0/3777915974 >> v2:172.21.15.32:3300/0 conn(0x7f1e9c0cb9a0 msgr2=0x7f1e9c0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:26:58.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1ea1f22700 1 --2- 172.21.15.3:0/3777915974 >> v2:172.21.15.32:3300/0 conn(0x7f1e9c0cb9a0 0x7f1e9c0cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1ea1f22700 1 -- 172.21.15.3:0/3777915974 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c0cd130 con 0x7f1e9c0709b0 2023-03-03T06:26:58.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1ea1f22700 1 --2- 172.21.15.3:0/3777915974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e9c0709b0 0x7f1e9c070e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0b1460 tx=0x7f1e9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ef18b384c2c1a7 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:58.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1e93fff700 1 -- 172.21.15.3:0/3777915974 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9400e070 con 0x7f1e9c0709b0 2023-03-03T06:26:58.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1e93fff700 1 -- 172.21.15.3:0/3777915974 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9400b8d0 con 0x7f1e9c0709b0 2023-03-03T06:26:58.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1e93fff700 1 -- 172.21.15.3:0/3777915974 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9400bc80 con 0x7f1e9c0709b0 2023-03-03T06:26:58.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1ea3985700 1 -- 172.21.15.3:0/3777915974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e9c0709b0 msgr2=0x7f1e9c070e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:58.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1ea3985700 1 --2- 172.21.15.3:0/3777915974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e9c0709b0 0x7f1e9c070e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0b1460 tx=0x7f1e9400b330 comp rx=0 tx=0).stop 2023-03-03T06:26:58.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1ea3985700 1 -- 172.21.15.3:0/3777915974 shutdown_connections 2023-03-03T06:26:58.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1ea3985700 1 --2- 172.21.15.3:0/3777915974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e9c0709b0 0x7f1e9c070e30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1ea3985700 1 --2- 172.21.15.3:0/3777915974 >> v2:172.21.15.32:3300/0 conn(0x7f1e9c0cb9a0 0x7f1e9c0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1ea3985700 1 --2- 172.21.15.3:0/3777915974 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e9c158400 0x7f1e9c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.802+0000 7f1ea3985700 1 -- 172.21.15.3:0/3777915974 >> 172.21.15.3:0/3777915974 conn(0x7f1e9c0607c0 msgr2=0x7f1e9c060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:26:58.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.806+0000 7f1ea3985700 1 -- 172.21.15.3:0/3777915974 shutdown_connections 2023-03-03T06:26:58.811 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.806+0000 7f1ea3985700 1 -- 172.21.15.3:0/3777915974 wait complete. 2023-03-03T06:26:58.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea3985700 1 Processor -- start 2023-03-03T06:26:58.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea3985700 1 -- start start 2023-03-03T06:26:58.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea3985700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e9c0709b0 0x7f1e9c151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:58.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea3985700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e9c0cb9a0 0x7f1e9c06bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:58.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea3985700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e9c158400 0x7f1e9c06c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:58.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea3985700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c1522c0 con 0x7f1e9c0cb9a0 2023-03-03T06:26:58.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea3985700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c15a9e0 con 0x7f1e9c158400 2023-03-03T06:26:58.816 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea3985700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c15a550 con 0x7f1e9c0709b0 2023-03-03T06:26:58.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea0f20700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e9c0cb9a0 0x7f1e9c06bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:58.817 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea0f20700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e9c0cb9a0 0x7f1e9c06bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52352/0 (socket says 172.21.15.3:52352) 2023-03-03T06:26:58.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea0f20700 1 -- 172.21.15.3:0/1439560560 learned_addr learned my addr 172.21.15.3:0/1439560560 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:26:58.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea1721700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e9c0709b0 0x7f1e9c151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:58.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea1f22700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e9c158400 0x7f1e9c06c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:58.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea0f20700 1 -- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e9c0709b0 msgr2=0x7f1e9c151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:58.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea0f20700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e9c0709b0 0x7f1e9c151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea0f20700 1 -- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e9c158400 msgr2=0x7f1e9c06c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:26:58.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.810+0000 7f1ea0f20700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e9c158400 0x7f1e9c06c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:26:58.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.814+0000 7f1ea0f20700 1 -- 172.21.15.3:0/1439560560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e94009b30 con 0x7f1e9c0cb9a0 2023-03-03T06:26:58.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.814+0000 7f1ea0f20700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e9c0cb9a0 0x7f1e9c06bf90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00e9d0 tx=0x7f1e8c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:58.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.814+0000 7f1e91ffb700 1 -- 172.21.15.3:0/1439560560 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c009720 con 0x7f1e9c0cb9a0 2023-03-03T06:26:58.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.814+0000 7f1e91ffb700 1 -- 172.21.15.3:0/1439560560 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8c00ee70 con 0x7f1e9c0cb9a0 2023-03-03T06:26:58.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.814+0000 7f1e91ffb700 1 -- 172.21.15.3:0/1439560560 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c00ca50 con 0x7f1e9c0cb9a0 2023-03-03T06:26:58.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.814+0000 7f1ea3985700 1 -- 172.21.15.3:0/1439560560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c15c4f0 con 0x7f1e9c0cb9a0 2023-03-03T06:26:58.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.814+0000 7f1ea3985700 1 -- 172.21.15.3:0/1439560560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c0cbf60 con 0x7f1e9c0cb9a0 2023-03-03T06:26:58.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.814+0000 7f1ea3985700 1 -- 172.21.15.3:0/1439560560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9c15c7d0 con 0x7f1e9c0cb9a0 2023-03-03T06:26:58.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.818+0000 7f1e91ffb700 1 -- 172.21.15.3:0/1439560560 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1e8c019030 con 0x7f1e9c0cb9a0 2023-03-03T06:26:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.822+0000 7f1e91ffb700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1e80064ee0 0x7f1e80067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:26:58.826 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.822+0000 7f1e91ffb700 1 -- 172.21.15.3:0/1439560560 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1e8c081f70 con 0x7f1e9c0cb9a0 2023-03-03T06:26:58.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.822+0000 7f1ea1721700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1e80064ee0 0x7f1e80067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:26:58.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.822+0000 7f1ea1721700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1e80064ee0 0x7f1e80067390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009ad0 tx=0x7f1e98009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:26:58.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:58.826+0000 7f1e91ffb700 1 -- 172.21.15.3:0/1439560560 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1e8c04fee0 con 0x7f1e9c0cb9a0 2023-03-03T06:26:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:26:59.038+0000 7f1ea3985700 1 -- 172.21.15.3:0/1439560560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1e9c073e90 con 0x7f1e9c0cb9a0 2023-03-03T06:27:00.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.006+0000 7f1e91ffb700 1 -- 172.21.15.3:0/1439560560 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1e8c053500 con 0x7f1e9c0cb9a0 2023-03-03T06:27:00.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.010+0000 7f1ea3985700 1 -- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1e80064ee0 msgr2=0x7f1e80067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:00.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.010+0000 7f1ea3985700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1e80064ee0 0x7f1e80067390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009ad0 tx=0x7f1e98009240 comp rx=0 tx=0).stop 2023-03-03T06:27:00.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.010+0000 7f1ea3985700 1 -- 172.21.15.3:0/1439560560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e9c0cb9a0 msgr2=0x7f1e9c06bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:00.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.010+0000 7f1ea3985700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e9c0cb9a0 0x7f1e9c06bf90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00e9d0 tx=0x7f1e8c00ece0 comp rx=0 tx=0).stop 2023-03-03T06:27:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.010+0000 7f1ea3985700 1 -- 172.21.15.3:0/1439560560 shutdown_connections 2023-03-03T06:27:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.010+0000 7f1ea3985700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1e80064ee0 0x7f1e80067390 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.010+0000 7f1ea3985700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e9c158400 0x7f1e9c06c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.010+0000 7f1ea3985700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e9c0cb9a0 0x7f1e9c06bf90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.010+0000 7f1ea3985700 1 --2- 172.21.15.3:0/1439560560 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1e9c0709b0 0x7f1e9c151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.010+0000 7f1ea3985700 1 -- 172.21.15.3:0/1439560560 >> 172.21.15.3:0/1439560560 conn(0x7f1e9c0607c0 msgr2=0x7f1e9c1559b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:00.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.010+0000 7f1ea3985700 1 -- 172.21.15.3:0/1439560560 shutdown_connections 2023-03-03T06:27:00.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.010+0000 7f1ea3985700 1 -- 172.21.15.3:0/1439560560 wait complete. 2023-03-03T06:27:00.049 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-03-03T06:27:00.050 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-03-03T06:27:00.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.262+0000 7f6a194fd700 1 Processor -- start 2023-03-03T06:27:00.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.262+0000 7f6a194fd700 1 -- start start 2023-03-03T06:27:00.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.262+0000 7f6a194fd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6a1406a780 0x7f6a140cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:00.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.262+0000 7f6a194fd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1415c230 con 0x7f6a140710c0 2023-03-03T06:27:00.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.262+0000 7f6a194fd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a140cd130 con 0x7f6a140cb9a0 2023-03-03T06:27:00.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.262+0000 7f6a194fd700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a140caac0 con 0x7f6a1406a780 2023-03-03T06:27:00.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.262+0000 7f6a18cfc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6a140710c0 0x7f6a140ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51678/0 (socket says 172.21.15.3:51678) 2023-03-03T06:27:00.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.262+0000 7f6a18cfc700 1 -- 172.21.15.3:0/3201415202 learned_addr learned my addr 172.21.15.3:0/3201415202 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:00.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.262+0000 7f6a0b7fe700 1 --1- 172.21.15.3:0/3201415202 >> v1:172.21.15.32:6789/0 conn(0x7f6a140cb9a0 0x7f6a140709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:54718/0 (socket says 172.21.15.3:54718) 2023-03-03T06:27:00.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.262+0000 7f6a13fff700 1 --2- 172.21.15.3:0/3201415202 >> v2:172.21.15.3:3300/0 conn(0x7f6a1406a780 0x7f6a140cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:00.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.266+0000 7f6a137fe700 1 -- 172.21.15.3:0/3201415202 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611176096 0 0) 0x7f6a140cd130 con 0x7f6a140cb9a0 2023-03-03T06:27:00.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.266+0000 7f6a137fe700 1 -- 172.21.15.3:0/3201415202 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a000035d0 con 0x7f6a140cb9a0 2023-03-03T06:27:00.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.266+0000 7f6a13fff700 1 -- 172.21.15.3:0/3201415202 >> v1:172.21.15.32:6789/0 conn(0x7f6a140cb9a0 legacy=0x7f6a140709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:00.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.266+0000 7f6a13fff700 1 -- 172.21.15.3:0/3201415202 >> v1:172.21.15.3:6789/0 conn(0x7f6a140710c0 legacy=0x7f6a140ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:00.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.266+0000 7f6a137fe700 1 -- 172.21.15.3:0/3201415202 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537501128 0 0) 0x7f6a1415c230 con 0x7f6a140710c0 2023-03-03T06:27:00.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.266+0000 7f6a13fff700 1 -- 172.21.15.3:0/3201415202 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a140cbf60 con 0x7f6a1406a780 2023-03-03T06:27:00.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.266+0000 7f6a13fff700 1 --2- 172.21.15.3:0/3201415202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a1406a780 0x7f6a140cace0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009850 tx=0x7f6a04009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed192eba6c34b6c0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:00.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.266+0000 7f6a137fe700 1 -- 172.21.15.3:0/3201415202 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04011830 con 0x7f6a1406a780 2023-03-03T06:27:00.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.266+0000 7f6a137fe700 1 -- 172.21.15.3:0/3201415202 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a04011990 con 0x7f6a1406a780 2023-03-03T06:27:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.266+0000 7f6a137fe700 1 -- 172.21.15.3:0/3201415202 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04011d40 con 0x7f6a1406a780 2023-03-03T06:27:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.266+0000 7f6a194fd700 1 -- 172.21.15.3:0/3201415202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a1406a780 msgr2=0x7f6a140cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.266+0000 7f6a194fd700 1 --2- 172.21.15.3:0/3201415202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a1406a780 0x7f6a140cace0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009850 tx=0x7f6a04009b60 comp rx=0 tx=0).stop 2023-03-03T06:27:00.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.270+0000 7f6a194fd700 1 -- 172.21.15.3:0/3201415202 shutdown_connections 2023-03-03T06:27:00.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.270+0000 7f6a194fd700 1 --2- 172.21.15.3:0/3201415202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a1406a780 0x7f6a140cace0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:00.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.270+0000 7f6a194fd700 1 -- 172.21.15.3:0/3201415202 >> 172.21.15.3:0/3201415202 conn(0x7f6a140607c0 msgr2=0x7f6a14060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:00.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.270+0000 7f6a194fd700 1 -- 172.21.15.3:0/3201415202 shutdown_connections 2023-03-03T06:27:00.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.270+0000 7f6a194fd700 1 -- 172.21.15.3:0/3201415202 wait complete. 2023-03-03T06:27:00.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.270+0000 7f6a194fd700 1 Processor -- start 2023-03-03T06:27:00.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.270+0000 7f6a194fd700 1 -- start start 2023-03-03T06:27:00.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a194fd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a1406a780 0x7f6a140b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:00.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a194fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a140710c0 0x7f6a14154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:00.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a194fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a140cb9a0 0x7f6a141f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:00.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a194fd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14153670 con 0x7f6a1406a780 2023-03-03T06:27:00.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a194fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140b3cb0 con 0x7f6a140cb9a0 2023-03-03T06:27:00.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a194fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a141f9300 con 0x7f6a140710c0 2023-03-03T06:27:00.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a13fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a1406a780 0x7f6a140b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:00.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a13fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a1406a780 0x7f6a140b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33610/0 (socket says 172.21.15.3:33610) 2023-03-03T06:27:00.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a13fff700 1 -- 172.21.15.3:0/535777894 learned_addr learned my addr 172.21.15.3:0/535777894 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:00.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a18cfc700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a140cb9a0 0x7f6a141f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:00.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a0b7fe700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a140710c0 0x7f6a14154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:00.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a13fff700 1 -- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a140710c0 msgr2=0x7f6a14154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:00.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a13fff700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a140710c0 0x7f6a14154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:00.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a13fff700 1 -- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a140cb9a0 msgr2=0x7f6a141f8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:00.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a13fff700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a140cb9a0 0x7f6a141f8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:00.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a13fff700 1 -- 172.21.15.3:0/535777894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a040094b0 con 0x7f6a1406a780 2023-03-03T06:27:00.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a13fff700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a1406a780 0x7f6a140b4140 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009eb0 tx=0x7f6a04004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:00.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a117fa700 1 -- 172.21.15.3:0/535777894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0401aba0 con 0x7f6a1406a780 2023-03-03T06:27:00.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a117fa700 1 -- 172.21.15.3:0/535777894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0401f070 con 0x7f6a1406a780 2023-03-03T06:27:00.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a117fa700 1 -- 172.21.15.3:0/535777894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0401b430 con 0x7f6a1406a780 2023-03-03T06:27:00.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.274+0000 7f6a194fd700 1 -- 172.21.15.3:0/535777894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1415c7d0 con 0x7f6a1406a780 2023-03-03T06:27:00.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.278+0000 7f6a194fd700 1 -- 172.21.15.3:0/535777894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1415c4f0 con 0x7f6a1406a780 2023-03-03T06:27:00.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.278+0000 7f6a194fd700 1 -- 172.21.15.3:0/535777894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1415cab0 con 0x7f6a1406a780 2023-03-03T06:27:00.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.278+0000 7f6a117fa700 1 -- 172.21.15.3:0/535777894 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6a0400f050 con 0x7f6a1406a780 2023-03-03T06:27:00.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.282+0000 7f6a117fa700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f69f4064ee0 0x7f69f4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:00.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.282+0000 7f6a117fa700 1 -- 172.21.15.3:0/535777894 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6a04083230 con 0x7f6a1406a780 2023-03-03T06:27:00.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.282+0000 7f6a0b7fe700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f69f4064ee0 0x7f69f4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:00.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.282+0000 7f6a0b7fe700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f69f4064ee0 0x7f69f4067390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6a14168aa0 tx=0x7f69fc007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:00.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.286+0000 7f6a117fa700 1 -- 172.21.15.3:0/535777894 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6a040511a0 con 0x7f6a1406a780 2023-03-03T06:27:00.504 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:00.498+0000 7f6a194fd700 1 -- 172.21.15.3:0/535777894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6a14073e90 con 0x7f6a1406a780 2023-03-03T06:27:01.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.018+0000 7f6a117fa700 1 -- 172.21.15.3:0/535777894 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6a040210c0 con 0x7f6a1406a780 2023-03-03T06:27:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.022+0000 7f6a194fd700 1 -- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f69f4064ee0 msgr2=0x7f69f4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.022+0000 7f6a194fd700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f69f4064ee0 0x7f69f4067390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6a14168aa0 tx=0x7f69fc007c00 comp rx=0 tx=0).stop 2023-03-03T06:27:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.022+0000 7f6a194fd700 1 -- 172.21.15.3:0/535777894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a1406a780 msgr2=0x7f6a140b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.022+0000 7f6a194fd700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a1406a780 0x7f6a140b4140 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009eb0 tx=0x7f6a04004770 comp rx=0 tx=0).stop 2023-03-03T06:27:01.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.026+0000 7f6a194fd700 1 -- 172.21.15.3:0/535777894 shutdown_connections 2023-03-03T06:27:01.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.026+0000 7f6a194fd700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f69f4064ee0 0x7f69f4067390 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.026+0000 7f6a194fd700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a140cb9a0 0x7f6a141f8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.026+0000 7f6a194fd700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6a140710c0 0x7f6a14154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.026+0000 7f6a194fd700 1 --2- 172.21.15.3:0/535777894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a1406a780 0x7f6a140b4140 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.026+0000 7f6a194fd700 1 -- 172.21.15.3:0/535777894 >> 172.21.15.3:0/535777894 conn(0x7f6a140607c0 msgr2=0x7f6a140709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:01.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.026+0000 7f6a194fd700 1 -- 172.21.15.3:0/535777894 shutdown_connections 2023-03-03T06:27:01.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.026+0000 7f6a194fd700 1 -- 172.21.15.3:0/535777894 wait complete. 2023-03-03T06:27:01.067 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-03-03T06:27:01.067 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-03T06:27:01.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.278+0000 7feeaa66f700 1 Processor -- start 2023-03-03T06:27:01.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.278+0000 7feeaa66f700 1 -- start start 2023-03-03T06:27:01.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feeaa66f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feea40c9f60 0x7feea40cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:01.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feeaa66f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feea40cd3a0 0x7feea40cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:01.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feeaa66f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea40ce100 con 0x7feea415c510 2023-03-03T06:27:01.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feeaa66f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea40ce450 con 0x7feea40c9f60 2023-03-03T06:27:01.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feeaa66f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea40ce730 con 0x7feea40cd3a0 2023-03-03T06:27:01.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feea966d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feea415c510 0x7feea40c9850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51680/0 (socket says 172.21.15.3:51680) 2023-03-03T06:27:01.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feea966d700 1 -- 172.21.15.3:0/2977661136 learned_addr learned my addr 172.21.15.3:0/2977661136 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feea8e6c700 1 --2- 172.21.15.3:0/2977661136 >> v2:172.21.15.3:3300/0 conn(0x7feea40c9f60 0x7feea40cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feea9e6e700 1 --2- 172.21.15.3:0/2977661136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feea40cd3a0 0x7feea40cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feea8e6c700 1 -- 172.21.15.3:0/2977661136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feea40cd3a0 msgr2=0x7feea40cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feea8e6c700 1 --2- 172.21.15.3:0/2977661136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feea40cd3a0 0x7feea40cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feea8e6c700 1 -- 172.21.15.3:0/2977661136 >> v1:172.21.15.3:6789/0 conn(0x7feea415c510 legacy=0x7feea40c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feea8e6c700 1 -- 172.21.15.3:0/2977661136 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea40cea10 con 0x7feea40c9f60 2023-03-03T06:27:01.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7feea8e6c700 1 --2- 172.21.15.3:0/2977661136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea40c9f60 0x7feea40cf210 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fee94009850 tx=0x7fee94009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526e25a3b11ae6e9 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7fee9bfff700 1 -- 172.21.15.3:0/2977661136 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94011830 con 0x7feea40c9f60 2023-03-03T06:27:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7fee9bfff700 1 -- 172.21.15.3:0/2977661136 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee94011990 con 0x7feea40c9f60 2023-03-03T06:27:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.282+0000 7fee9bfff700 1 -- 172.21.15.3:0/2977661136 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94011d40 con 0x7feea40c9f60 2023-03-03T06:27:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.286+0000 7feeaa66f700 1 -- 172.21.15.3:0/2977661136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea40c9f60 msgr2=0x7feea40cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.286+0000 7feeaa66f700 1 --2- 172.21.15.3:0/2977661136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea40c9f60 0x7feea40cf210 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fee94009850 tx=0x7fee94009b60 comp rx=0 tx=0).stop 2023-03-03T06:27:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.286+0000 7feeaa66f700 1 -- 172.21.15.3:0/2977661136 shutdown_connections 2023-03-03T06:27:01.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.286+0000 7feeaa66f700 1 --2- 172.21.15.3:0/2977661136 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feea40cd3a0 0x7feea40cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.286+0000 7feeaa66f700 1 --2- 172.21.15.3:0/2977661136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea40c9f60 0x7feea40cf210 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.286+0000 7feeaa66f700 1 -- 172.21.15.3:0/2977661136 >> 172.21.15.3:0/2977661136 conn(0x7feea4070a30 msgr2=0x7feea4070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:01.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.286+0000 7feeaa66f700 1 -- 172.21.15.3:0/2977661136 shutdown_connections 2023-03-03T06:27:01.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.286+0000 7feeaa66f700 1 -- 172.21.15.3:0/2977661136 wait complete. 2023-03-03T06:27:01.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.290+0000 7feeaa66f700 1 Processor -- start 2023-03-03T06:27:01.296 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.290+0000 7feeaa66f700 1 -- start start 2023-03-03T06:27:01.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feeaa66f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea40c9f60 0x7feea40aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:01.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feeaa66f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feea40cd3a0 0x7feea41578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:01.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feeaa66f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea415c510 0x7feea4157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:01.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feeaa66f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea4159200 con 0x7feea40c9f60 2023-03-03T06:27:01.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feeaa66f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea4158340 con 0x7feea415c510 2023-03-03T06:27:01.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feeaa66f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea41585f0 con 0x7feea40cd3a0 2023-03-03T06:27:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feea966d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea40c9f60 0x7feea40aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feea966d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea40c9f60 0x7feea40aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33630/0 (socket says 172.21.15.3:33630) 2023-03-03T06:27:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feea966d700 1 -- 172.21.15.3:0/3743601104 learned_addr learned my addr 172.21.15.3:0/3743601104 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feea8e6c700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feea40cd3a0 0x7feea41578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:01.301 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feea9e6e700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea415c510 0x7feea4157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feea966d700 1 -- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feea40cd3a0 msgr2=0x7feea41578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feea966d700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feea40cd3a0 0x7feea41578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feea966d700 1 -- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea415c510 msgr2=0x7feea4157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feea966d700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea415c510 0x7feea4157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feea966d700 1 -- 172.21.15.3:0/3743601104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee940094b0 con 0x7feea40c9f60 2023-03-03T06:27:01.302 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feea966d700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea40c9f60 0x7feea40aca40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feea0001eb0 tx=0x7feea000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:01.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7fee99ffb700 1 -- 172.21.15.3:0/3743601104 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0005d90 con 0x7feea40c9f60 2023-03-03T06:27:01.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7fee99ffb700 1 -- 172.21.15.3:0/3743601104 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea00070d0 con 0x7feea40c9f60 2023-03-03T06:27:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7fee99ffb700 1 -- 172.21.15.3:0/3743601104 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0007440 con 0x7feea40c9f60 2023-03-03T06:27:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feeaa66f700 1 -- 172.21.15.3:0/3743601104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea415c8e0 con 0x7feea40c9f60 2023-03-03T06:27:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.294+0000 7feeaa66f700 1 -- 172.21.15.3:0/3743601104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea40cecf0 con 0x7feea40c9f60 2023-03-03T06:27:01.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.298+0000 7feeaa66f700 1 -- 172.21.15.3:0/3743601104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea40cdd60 con 0x7feea40c9f60 2023-03-03T06:27:01.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.298+0000 7fee99ffb700 1 -- 172.21.15.3:0/3743601104 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7feea0019030 con 0x7feea40c9f60 2023-03-03T06:27:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.302+0000 7fee99ffb700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fee88064f10 0x7fee880673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:01.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.302+0000 7fee99ffb700 1 -- 172.21.15.3:0/3743601104 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7feea0081220 con 0x7feea40c9f60 2023-03-03T06:27:01.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.302+0000 7feea8e6c700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fee88064f10 0x7fee880673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:01.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.302+0000 7feea8e6c700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fee88064f10 0x7fee880673c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feea40ca350 tx=0x7fee94008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:01.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.310+0000 7fee99ffb700 1 -- 172.21.15.3:0/3743601104 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7feea004a400 con 0x7feea40c9f60 2023-03-03T06:27:01.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.518+0000 7feeaa66f700 1 -- 172.21.15.3:0/3743601104 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feea4073e90 con 0x7feea40c9f60 2023-03-03T06:27:01.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.518+0000 7fee99ffb700 1 -- 172.21.15.3:0/3743601104 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4643 (secure 0 0 0) 0x7feea004f240 con 0x7feea40c9f60 2023-03-03T06:27:01.525 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:27:01.525 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5358,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/3530112681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3530112681},{"type":"v1","addr":"172.21.15.3:6833","nonce":3530112681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2160506868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2160506868},{"type":"v1","addr":"172.21.15.3:6835","nonce":2160506868}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":54887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/4289059366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4289059366},{"type":"v1","addr":"172.21.15.32:6836","nonce":4289059366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-03T06:26:57.500775+0000","modified":"2023-03-03T06:27:01.007679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54890":{"gid":54890,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.32:6837/947618180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":947618180},{"type":"v1","addr":"172.21.15.32:6837","nonce":947618180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-03-03T06:27:01.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.522+0000 7feeaa66f700 1 -- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fee88064f10 msgr2=0x7fee880673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.522+0000 7feeaa66f700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fee88064f10 0x7fee880673c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feea40ca350 tx=0x7fee94008030 comp rx=0 tx=0).stop 2023-03-03T06:27:01.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.522+0000 7feeaa66f700 1 -- 172.21.15.3:0/3743601104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea40c9f60 msgr2=0x7feea40aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.522+0000 7feeaa66f700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea40c9f60 0x7feea40aca40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feea0001eb0 tx=0x7feea000c7b0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.522+0000 7feeaa66f700 1 -- 172.21.15.3:0/3743601104 shutdown_connections 2023-03-03T06:27:01.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.522+0000 7feeaa66f700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fee88064f10 0x7fee880673c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.522+0000 7feeaa66f700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feea415c510 0x7feea4157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.522+0000 7feeaa66f700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7feea40cd3a0 0x7feea41578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.522+0000 7feeaa66f700 1 --2- 172.21.15.3:0/3743601104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feea40c9f60 0x7feea40aca40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.522+0000 7feeaa66f700 1 -- 172.21.15.3:0/3743601104 >> 172.21.15.3:0/3743601104 conn(0x7feea4070a30 msgr2=0x7feea4070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:01.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.522+0000 7feeaa66f700 1 -- 172.21.15.3:0/3743601104 shutdown_connections 2023-03-03T06:27:01.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.522+0000 7feeaa66f700 1 -- 172.21.15.3:0/3743601104 wait complete. 2023-03-03T06:27:01.533 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 17 2023-03-03T06:27:01.560 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-03T06:27:01.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.754+0000 7fb9b0809700 1 Processor -- start 2023-03-03T06:27:01.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.754+0000 7fb9b0809700 1 -- start start 2023-03-03T06:27:01.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.758+0000 7fb9b0809700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9ac0c9f60 0x7fb9ac0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:01.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.758+0000 7fb9b0809700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb9ac0cd3a0 0x7fb9ac0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:01.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.758+0000 7fb9b0809700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ac0ce100 con 0x7fb9ac15c510 2023-03-03T06:27:01.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.758+0000 7fb9b0809700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ac0ce450 con 0x7fb9ac0cd3a0 2023-03-03T06:27:01.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.758+0000 7fb9b0809700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ac0ce730 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.758+0000 7fb9ab59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb9ac0cd3a0 0x7fb9ac0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:01.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.758+0000 7fb9aa59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9ac0c9f60 0x7fb9ac0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:01.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.758+0000 7fb9ab59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb9ac0cd3a0 0x7fb9ac0cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33644/0 (socket says 172.21.15.3:33644) 2023-03-03T06:27:01.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.758+0000 7fb9ab59e700 1 -- 172.21.15.3:0/3904519314 learned_addr learned my addr 172.21.15.3:0/3904519314 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:01.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.758+0000 7fb9ab59e700 1 -- 172.21.15.3:0/3904519314 >> v2:172.21.15.32:3300/0 conn(0x7fb9ac0c9f60 msgr2=0x7fb9ac0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:01.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.758+0000 7fb9ab59e700 1 --2- 172.21.15.3:0/3904519314 >> v2:172.21.15.32:3300/0 conn(0x7fb9ac0c9f60 0x7fb9ac0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.758+0000 7fb9ab59e700 1 -- 172.21.15.3:0/3904519314 >> v1:172.21.15.32:6789/0 conn(0x7fb9ac15c510 legacy=0x7fb9ac0c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.758+0000 7fb9ab59e700 1 -- 172.21.15.3:0/3904519314 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac0cea10 con 0x7fb9ac0cd3a0 2023-03-03T06:27:01.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.762+0000 7fb9ab59e700 1 --2- 172.21.15.3:0/3904519314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac0cd3a0 0x7fb9ac0cd820 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0cca70 tx=0x7fb9a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f33489362ff5da server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:01.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.762+0000 7fb9a9d9b700 1 -- 172.21.15.3:0/3904519314 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a400e070 con 0x7fb9ac0cd3a0 2023-03-03T06:27:01.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.762+0000 7fb9a9d9b700 1 -- 172.21.15.3:0/3904519314 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a400b910 con 0x7fb9ac0cd3a0 2023-03-03T06:27:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.762+0000 7fb9a9d9b700 1 -- 172.21.15.3:0/3904519314 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a400bcc0 con 0x7fb9ac0cd3a0 2023-03-03T06:27:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.762+0000 7fb9b0809700 1 -- 172.21.15.3:0/3904519314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac0cd3a0 msgr2=0x7fb9ac0cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.762+0000 7fb9b0809700 1 --2- 172.21.15.3:0/3904519314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac0cd3a0 0x7fb9ac0cd820 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0cca70 tx=0x7fb9a400b330 comp rx=0 tx=0).stop 2023-03-03T06:27:01.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.762+0000 7fb9b0809700 1 -- 172.21.15.3:0/3904519314 shutdown_connections 2023-03-03T06:27:01.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.762+0000 7fb9b0809700 1 --2- 172.21.15.3:0/3904519314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac0cd3a0 0x7fb9ac0cd820 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.762+0000 7fb9b0809700 1 --2- 172.21.15.3:0/3904519314 >> v2:172.21.15.32:3300/0 conn(0x7fb9ac0c9f60 0x7fb9ac0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.762+0000 7fb9b0809700 1 -- 172.21.15.3:0/3904519314 >> 172.21.15.3:0/3904519314 conn(0x7fb9ac070a30 msgr2=0x7fb9ac070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:01.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.762+0000 7fb9b0809700 1 -- 172.21.15.3:0/3904519314 shutdown_connections 2023-03-03T06:27:01.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.762+0000 7fb9b0809700 1 -- 172.21.15.3:0/3904519314 wait complete. 2023-03-03T06:27:01.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.766+0000 7fb9b0809700 1 Processor -- start 2023-03-03T06:27:01.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.766+0000 7fb9b0809700 1 -- start start 2023-03-03T06:27:01.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9b0809700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb9ac0c9f60 0x7fb9ac0c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:01.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9b0809700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ac0cd3a0 0x7fb9ac147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:01.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9b0809700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac15c510 0x7fb9ac0abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:01.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9b0809700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0c7560 con 0x7fb9ac15c510 2023-03-03T06:27:01.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9b0809700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0c7ec0 con 0x7fb9ac0cd3a0 2023-03-03T06:27:01.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9b0809700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1480f0 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9ab59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac15c510 0x7fb9ac0abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:01.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9aad9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb9ac0c9f60 0x7fb9ac0c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:01.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9ab59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac15c510 0x7fb9ac0abfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33658/0 (socket says 172.21.15.3:33658) 2023-03-03T06:27:01.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9ab59e700 1 -- 172.21.15.3:0/2013524223 learned_addr learned my addr 172.21.15.3:0/2013524223 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:01.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9aa59c700 1 --2- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ac0cd3a0 0x7fb9ac147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:01.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9aad9d700 1 -- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ac0cd3a0 msgr2=0x7fb9ac147bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9aad9d700 1 --2- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ac0cd3a0 0x7fb9ac147bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9aad9d700 1 -- 172.21.15.3:0/2013524223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac15c510 msgr2=0x7fb9ac0abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9aad9d700 1 --2- 172.21.15.3:0/2013524223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac15c510 0x7fb9ac0abfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9aad9d700 1 -- 172.21.15.3:0/2013524223 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a4009b30 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9aad9d700 1 --2- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb9ac0c9f60 0x7fb9ac0c7980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00c4b0 tx=0x7fb99c00c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:01.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9937fe700 1 -- 172.21.15.3:0/2013524223 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0052d0 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9937fe700 1 -- 172.21.15.3:0/2013524223 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c005430 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9b0809700 1 -- 172.21.15.3:0/2013524223 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac15c8e0 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9937fe700 1 -- 172.21.15.3:0/2013524223 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0055a0 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.770+0000 7fb9b0809700 1 -- 172.21.15.3:0/2013524223 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac0cecf0 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.774+0000 7fb9b0809700 1 -- 172.21.15.3:0/2013524223 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ac0cdd60 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.780 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.774+0000 7fb9937fe700 1 -- 172.21.15.3:0/2013524223 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb99c019030 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.778+0000 7fb9937fe700 1 --2- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb998064fb0 0x7fb998067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:01.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.778+0000 7fb9937fe700 1 -- 172.21.15.3:0/2013524223 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb99c081290 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.778+0000 7fb9aa59c700 1 --2- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb998064fb0 0x7fb998067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:01.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.778+0000 7fb9aa59c700 1 --2- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb998064fb0 0x7fb998067460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0adf80 tx=0x7fb9a0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:01.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.786+0000 7fb9937fe700 1 -- 172.21.15.3:0/2013524223 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb99c04f280 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.962+0000 7fb9b0809700 1 -- 172.21.15.3:0/2013524223 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9ac0c76a0 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.966+0000 7fb9937fe700 1 -- 172.21.15.3:0/2013524223 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17936 (secure 0 0 0) 0x7fb99c0528a0 con 0x7fb9ac0c9f60 2023-03-03T06:27:01.972 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:27:01.972 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":43,"fsid":"7d496254-7268-405f-a1d8-c8033dc8659a","created":"2023-03-03T06:24:44.867015+0000","modified":"2023-03-03T06:26:58.507015+0000","last_up_change":"2023-03-03T06:24:58.657332+0000","last_in_change":"2023-03-03T06:24:48.459752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-03T06:25:00.514768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-03T06:25:05.271759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-03-03T06:26:55.353616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-03-03T06:26:56.534268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"51e6c897-995b-4faf-bfb2-9e749ab1c989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":32269},{"type":"v1","addr":"172.21.15.3:6825","nonce":32269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":32269},{"type":"v1","addr":"172.21.15.3:6827","nonce":32269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":32269},{"type":"v1","addr":"172.21.15.3:6831","nonce":32269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":32269},{"type":"v1","addr":"172.21.15.3:6829","nonce":32269}]},"public_addr":"172.21.15.3:6825/32269","cluster_addr":"172.21.15.3:6827/32269","heartbeat_back_addr":"172.21.15.3:6831/32269","heartbeat_front_addr":"172.21.15.3:6829/32269","state":["exists","up"]},{"osd":1,"uuid":"b6f1d313-916a-420e-b37f-1f05848d0968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":32272},{"type":"v1","addr":"172.21.15.3:6809","nonce":32272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":32272},{"type":"v1","addr":"172.21.15.3:6811","nonce":32272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":32272},{"type":"v1","addr":"172.21.15.3:6815","nonce":32272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":32272},{"type":"v1","addr":"172.21.15.3:6813","nonce":32272}]},"public_addr":"172.21.15.3:6809/32272","cluster_addr":"172.21.15.3:6811/32272","heartbeat_back_addr":"172.21.15.3:6815/32272","heartbeat_front_addr":"172.21.15.3:6813/32272","state":["exists","up"]},{"osd":2,"uuid":"71d83c5c-1ae5-4441-a1de-d260cc415bcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":32271},{"type":"v1","addr":"172.21.15.3:6801","nonce":32271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":32271},{"type":"v1","addr":"172.21.15.3:6803","nonce":32271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":32271},{"type":"v1","addr":"172.21.15.3:6807","nonce":32271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":32271},{"type":"v1","addr":"172.21.15.3:6805","nonce":32271}]},"public_addr":"172.21.15.3:6801/32271","cluster_addr":"172.21.15.3:6803/32271","heartbeat_back_addr":"172.21.15.3:6807/32271","heartbeat_front_addr":"172.21.15.3:6805/32271","state":["exists","up"]},{"osd":3,"uuid":"b4cc8247-7d5e-4807-971c-28d6b7d7b128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":32268},{"type":"v1","addr":"172.21.15.3:6817","nonce":32268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":32268},{"type":"v1","addr":"172.21.15.3:6819","nonce":32268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":32268},{"type":"v1","addr":"172.21.15.3:6823","nonce":32268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":32268},{"type":"v1","addr":"172.21.15.3:6821","nonce":32268}]},"public_addr":"172.21.15.3:6817/32268","cluster_addr":"172.21.15.3:6819/32268","heartbeat_back_addr":"172.21.15.3:6823/32268","heartbeat_front_addr":"172.21.15.3:6821/32268","state":["exists","up"]},{"osd":4,"uuid":"3aea1cb1-e900-42d7-906a-cc8de12f00c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":32451},{"type":"v1","addr":"172.21.15.32:6803","nonce":32451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":32451},{"type":"v1","addr":"172.21.15.32:6809","nonce":32451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":32451},{"type":"v1","addr":"172.21.15.32:6815","nonce":32451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6811","nonce":32451},{"type":"v1","addr":"172.21.15.32:6813","nonce":32451}]},"public_addr":"172.21.15.32:6803/32451","cluster_addr":"172.21.15.32:6809/32451","heartbeat_back_addr":"172.21.15.32:6815/32451","heartbeat_front_addr":"172.21.15.32:6813/32451","state":["exists","up"]},{"osd":5,"uuid":"a7e1f2d6-bdce-403f-a0af-726bca7bca68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":32453},{"type":"v1","addr":"172.21.15.32:6825","nonce":32453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":32453},{"type":"v1","addr":"172.21.15.32:6827","nonce":32453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":32453},{"type":"v1","addr":"172.21.15.32:6831","nonce":32453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":32453},{"type":"v1","addr":"172.21.15.32:6829","nonce":32453}]},"public_addr":"172.21.15.32:6825/32453","cluster_addr":"172.21.15.32:6827/32453","heartbeat_back_addr":"172.21.15.32:6831/32453","heartbeat_front_addr":"172.21.15.32:6829/32453","state":["exists","up"]},{"osd":6,"uuid":"c395a30c-360d-408e-93b9-2819033f7725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6801","nonce":32452},{"type":"v1","addr":"172.21.15.32:6802","nonce":32452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":32452},{"type":"v1","addr":"172.21.15.32:6805","nonce":32452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":32452},{"type":"v1","addr":"172.21.15.32:6812","nonce":32452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6807","nonce":32452},{"type":"v1","addr":"172.21.15.32:6808","nonce":32452}]},"public_addr":"172.21.15.32:6802/32452","cluster_addr":"172.21.15.32:6805/32452","heartbeat_back_addr":"172.21.15.32:6812/32452","heartbeat_front_addr":"172.21.15.32:6808/32452","state":["exists","up"]},{"osd":7,"uuid":"db30b9e6-7cd4-4ceb-8555-8c8cca2b4e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":32454},{"type":"v1","addr":"172.21.15.32:6817","nonce":32454}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":32454},{"type":"v1","addr":"172.21.15.32:6819","nonce":32454}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":32454},{"type":"v1","addr":"172.21.15.32:6823","nonce":32454}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":32454},{"type":"v1","addr":"172.21.15.32:6821","nonce":32454}]},"public_addr":"172.21.15.32:6817/32454","cluster_addr":"172.21.15.32:6819/32454","heartbeat_back_addr":"172.21.15.32:6823/32454","heartbeat_front_addr":"172.21.15.32:6821/32454","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.634599+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.519150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.538927+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.574674+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.561829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.606925+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.496124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-03T06:24:54.646983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-03T06:27:01.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.970+0000 7fb9b0809700 1 -- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb998064fb0 msgr2=0x7fb998067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.970+0000 7fb9b0809700 1 --2- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb998064fb0 0x7fb998067460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0adf80 tx=0x7fb9a0009240 comp rx=0 tx=0).stop 2023-03-03T06:27:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.970+0000 7fb9b0809700 1 -- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb9ac0c9f60 msgr2=0x7fb9ac0c7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.970+0000 7fb9b0809700 1 --2- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb9ac0c9f60 0x7fb9ac0c7980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00c4b0 tx=0x7fb99c00c7c0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.970+0000 7fb9b0809700 1 -- 172.21.15.3:0/2013524223 shutdown_connections 2023-03-03T06:27:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.970+0000 7fb9b0809700 1 --2- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb998064fb0 0x7fb998067460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.970+0000 7fb9b0809700 1 --2- 172.21.15.3:0/2013524223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb9ac15c510 0x7fb9ac0abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.970+0000 7fb9b0809700 1 --2- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9ac0cd3a0 0x7fb9ac147bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.970+0000 7fb9b0809700 1 --2- 172.21.15.3:0/2013524223 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb9ac0c9f60 0x7fb9ac0c7980 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.970+0000 7fb9b0809700 1 -- 172.21.15.3:0/2013524223 >> 172.21.15.3:0/2013524223 conn(0x7fb9ac070a30 msgr2=0x7fb9ac15b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.970+0000 7fb9b0809700 1 -- 172.21.15.3:0/2013524223 shutdown_connections 2023-03-03T06:27:01.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:01.970+0000 7fb9b0809700 1 -- 172.21.15.3:0/2013524223 wait complete. 2023-03-03T06:27:02.010 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-03-03T06:27:02.011 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-03T06:27:02.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.210+0000 7fb78709f700 1 Processor -- start 2023-03-03T06:27:02.216 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.210+0000 7fb78709f700 1 -- start start 2023-03-03T06:27:02.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78709f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb78006db90 0x7fb7800cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:02.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78709f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7800cd410 con 0x7fb7800cb990 2023-03-03T06:27:02.218 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78709f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78006e130 con 0x7fb78006ecd0 2023-03-03T06:27:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78709f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800b12c0 con 0x7fb78006db90 2023-03-03T06:27:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78609d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb7800cb990 0x7fb78006e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51688/0 (socket says 172.21.15.3:51688) 2023-03-03T06:27:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78589c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb78006ecd0 0x7fb7800cf0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:54728/0 (socket says 172.21.15.3:54728) 2023-03-03T06:27:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78609d700 1 -- 172.21.15.3:0/2324055905 learned_addr learned my addr 172.21.15.3:0/2324055905 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:02.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78689e700 1 --2- 172.21.15.3:0/2324055905 >> v2:172.21.15.32:3300/0 conn(0x7fb78006db90 0x7fb7800cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:02.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78509b700 1 -- 172.21.15.3:0/2324055905 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631717560 0 0) 0x7fb78006e130 con 0x7fb78006ecd0 2023-03-03T06:27:02.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78509b700 1 -- 172.21.15.3:0/2324055905 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb76c0035d0 con 0x7fb78006ecd0 2023-03-03T06:27:02.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78509b700 1 -- 172.21.15.3:0/2324055905 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303765070 0 0) 0x7fb7800cd410 con 0x7fb7800cb990 2023-03-03T06:27:02.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78509b700 1 -- 172.21.15.3:0/2324055905 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb78006e130 con 0x7fb7800cb990 2023-03-03T06:27:02.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78689e700 1 -- 172.21.15.3:0/2324055905 >> v1:172.21.15.32:6789/0 conn(0x7fb78006ecd0 legacy=0x7fb7800cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:02.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78689e700 1 -- 172.21.15.3:0/2324055905 >> v1:172.21.15.3:6789/0 conn(0x7fb7800cb990 legacy=0x7fb78006e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:02.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.214+0000 7fb78689e700 1 -- 172.21.15.3:0/2324055905 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7800b15a0 con 0x7fb78006db90 2023-03-03T06:27:02.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.218+0000 7fb78689e700 1 --2- 172.21.15.3:0/2324055905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb78006db90 0x7fb7800cd8a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb778009850 tx=0x7fb778009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d275b9f714348ec4 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:02.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.218+0000 7fb78509b700 1 -- 172.21.15.3:0/2324055905 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778011830 con 0x7fb78006db90 2023-03-03T06:27:02.223 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.218+0000 7fb78509b700 1 -- 172.21.15.3:0/2324055905 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb778011990 con 0x7fb78006db90 2023-03-03T06:27:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.218+0000 7fb78509b700 1 -- 172.21.15.3:0/2324055905 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778011d40 con 0x7fb78006db90 2023-03-03T06:27:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.218+0000 7fb78709f700 1 -- 172.21.15.3:0/2324055905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb78006db90 msgr2=0x7fb7800cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.218+0000 7fb78709f700 1 --2- 172.21.15.3:0/2324055905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb78006db90 0x7fb7800cd8a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb778009850 tx=0x7fb778009b60 comp rx=0 tx=0).stop 2023-03-03T06:27:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.222+0000 7fb78709f700 1 -- 172.21.15.3:0/2324055905 shutdown_connections 2023-03-03T06:27:02.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.222+0000 7fb78709f700 1 --2- 172.21.15.3:0/2324055905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb78006db90 0x7fb7800cd8a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.222+0000 7fb78709f700 1 -- 172.21.15.3:0/2324055905 >> 172.21.15.3:0/2324055905 conn(0x7fb780069d70 msgr2=0x7fb78006a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:02.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.222+0000 7fb78709f700 1 -- 172.21.15.3:0/2324055905 shutdown_connections 2023-03-03T06:27:02.226 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.222+0000 7fb78709f700 1 -- 172.21.15.3:0/2324055905 wait complete. 2023-03-03T06:27:02.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78709f700 1 Processor -- start 2023-03-03T06:27:02.230 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78709f700 1 -- start start 2023-03-03T06:27:02.231 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78709f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb78006db90 0x7fb7800c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78709f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb78006ecd0 0x7fb78015b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78709f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb7800cb990 0x7fb7800c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78709f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800c8fe0 con 0x7fb78006ecd0 2023-03-03T06:27:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78709f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800c9280 con 0x7fb78006db90 2023-03-03T06:27:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78709f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800c3870 con 0x7fb7800cb990 2023-03-03T06:27:02.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78589c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb78006ecd0 0x7fb78015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:02.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78589c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb78006ecd0 0x7fb78015b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33664/0 (socket says 172.21.15.3:33664) 2023-03-03T06:27:02.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78589c700 1 -- 172.21.15.3:0/2879220048 learned_addr learned my addr 172.21.15.3:0/2879220048 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:02.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78609d700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb78006db90 0x7fb7800c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:02.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78689e700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb7800cb990 0x7fb7800c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:02.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78589c700 1 -- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb7800cb990 msgr2=0x7fb7800c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:02.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78589c700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb7800cb990 0x7fb7800c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78589c700 1 -- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb78006db90 msgr2=0x7fb7800c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:02.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78589c700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb78006db90 0x7fb7800c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78589c700 1 -- 172.21.15.3:0/2879220048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7780094b0 con 0x7fb78006ecd0 2023-03-03T06:27:02.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.226+0000 7fb78589c700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb78006ecd0 0x7fb78015b190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb77000c700 tx=0x7fb77000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:02.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.230+0000 7fb776ffd700 1 -- 172.21.15.3:0/2879220048 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770005cd0 con 0x7fb78006ecd0 2023-03-03T06:27:02.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.230+0000 7fb776ffd700 1 -- 172.21.15.3:0/2879220048 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb770005e30 con 0x7fb78006ecd0 2023-03-03T06:27:02.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.230+0000 7fb78709f700 1 -- 172.21.15.3:0/2879220048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7800cd080 con 0x7fb78006ecd0 2023-03-03T06:27:02.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.230+0000 7fb776ffd700 1 -- 172.21.15.3:0/2879220048 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7700072d0 con 0x7fb78006ecd0 2023-03-03T06:27:02.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.230+0000 7fb78709f700 1 -- 172.21.15.3:0/2879220048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7800b1880 con 0x7fb78006ecd0 2023-03-03T06:27:02.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.230+0000 7fb78709f700 1 -- 172.21.15.3:0/2879220048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7800cd210 con 0x7fb78006ecd0 2023-03-03T06:27:02.237 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.234+0000 7fb776ffd700 1 -- 172.21.15.3:0/2879220048 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb770019030 con 0x7fb78006ecd0 2023-03-03T06:27:02.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.234+0000 7fb776ffd700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb764064f10 0x7fb7640673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.234+0000 7fb776ffd700 1 -- 172.21.15.3:0/2879220048 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb770081430 con 0x7fb78006ecd0 2023-03-03T06:27:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.234+0000 7fb78609d700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb764064f10 0x7fb7640673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:02.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.238+0000 7fb78609d700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb764064f10 0x7fb7640673c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb77c001560 tx=0x7fb77c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:02.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.238+0000 7fb776ffd700 1 -- 172.21.15.3:0/2879220048 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb77004a400 con 0x7fb78006ecd0 2023-03-03T06:27:02.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.410+0000 7fb78709f700 1 -- 172.21.15.3:0/2879220048 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7800c36b0 con 0x7fb764064f10 2023-03-03T06:27:02.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.430+0000 7fb776ffd700 1 -- 172.21.15.3:0/2879220048 <== mgr.4105 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308653 (secure 0 0 0) 0x7fb7800c36b0 con 0x7fb764064f10 2023-03-03T06:27:02.440 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:27:02.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.442+0000 7fb78709f700 1 -- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb764064f10 msgr2=0x7fb7640673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:02.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.442+0000 7fb78709f700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb764064f10 0x7fb7640673c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb77c001560 tx=0x7fb77c009040 comp rx=0 tx=0).stop 2023-03-03T06:27:02.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.442+0000 7fb78709f700 1 -- 172.21.15.3:0/2879220048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb78006ecd0 msgr2=0x7fb78015b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:02.448 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.442+0000 7fb78709f700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb78006ecd0 0x7fb78015b190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb77000c700 tx=0x7fb77000cac0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.442+0000 7fb78709f700 1 -- 172.21.15.3:0/2879220048 shutdown_connections 2023-03-03T06:27:02.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.442+0000 7fb78709f700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fb764064f10 0x7fb7640673c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.442+0000 7fb78709f700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fb7800cb990 0x7fb7800c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.442+0000 7fb78709f700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb78006ecd0 0x7fb78015b190 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.442+0000 7fb78709f700 1 --2- 172.21.15.3:0/2879220048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb78006db90 0x7fb7800c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.449 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.442+0000 7fb78709f700 1 -- 172.21.15.3:0/2879220048 >> 172.21.15.3:0/2879220048 conn(0x7fb780069d70 msgr2=0x7fb78015feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:02.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.442+0000 7fb78709f700 1 -- 172.21.15.3:0/2879220048 shutdown_connections 2023-03-03T06:27:02.450 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.442+0000 7fb78709f700 1 -- 172.21.15.3:0/2879220048 wait complete. 2023-03-03T06:27:02.451 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-03-03T06:27:02.482 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-03-03T06:27:00.438970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":96652,"kb_used_data":5068,"kb_used_omap":0,"kb_used_meta":91520,"kb_avail":754878068,"statfs":{"total":773094113280,"available":772995141632,"internally_reserved":0,"allocated":5189632,"data_stored":2938107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":93716480},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.477701"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.554743+0000","last_change":"2023-03-03T06:26:57.550461+0000","last_active":"2023-03-03T06:26:57.554743+0000","last_peered":"2023-03-03T06:26:57.554743+0000","last_clean":"2023-03-03T06:26:57.554743+0000","last_became_active":"2023-03-03T06:26:57.550107+0000","last_became_peered":"2023-03-03T06:26:57.550107+0000","last_unstale":"2023-03-03T06:26:57.554743+0000","last_undegraded":"2023-03-03T06:26:57.554743+0000","last_fullsized":"2023-03-03T06:26:57.554743+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:25:57.269311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527519+0000","last_change":"2023-03-03T06:26:57.532776+0000","last_active":"2023-03-03T06:26:58.527519+0000","last_peered":"2023-03-03T06:26:58.527519+0000","last_clean":"2023-03-03T06:26:58.527519+0000","last_became_active":"2023-03-03T06:26:57.532061+0000","last_became_peered":"2023-03-03T06:26:57.532061+0000","last_unstale":"2023-03-03T06:26:58.527519+0000","last_undegraded":"2023-03-03T06:26:58.527519+0000","last_fullsized":"2023-03-03T06:26:58.527519+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:36:17.212390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526886+0000","last_change":"2023-03-03T06:26:57.530969+0000","last_active":"2023-03-03T06:26:58.526886+0000","last_peered":"2023-03-03T06:26:58.526886+0000","last_clean":"2023-03-03T06:26:58.526886+0000","last_became_active":"2023-03-03T06:26:57.530694+0000","last_became_peered":"2023-03-03T06:26:57.530694+0000","last_unstale":"2023-03-03T06:26:58.526886+0000","last_undegraded":"2023-03-03T06:26:58.526886+0000","last_fullsized":"2023-03-03T06:26:58.526886+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:00:05.544209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.550932+0000","last_change":"2023-03-03T06:26:57.543461+0000","last_active":"2023-03-03T06:26:57.550932+0000","last_peered":"2023-03-03T06:26:57.550932+0000","last_clean":"2023-03-03T06:26:57.550932+0000","last_became_active":"2023-03-03T06:26:57.543203+0000","last_became_peered":"2023-03-03T06:26:57.543203+0000","last_unstale":"2023-03-03T06:26:57.550932+0000","last_undegraded":"2023-03-03T06:26:57.550932+0000","last_fullsized":"2023-03-03T06:26:57.550932+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:33:49.811191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526662+0000","last_change":"2023-03-03T06:26:57.532765+0000","last_active":"2023-03-03T06:26:58.526662+0000","last_peered":"2023-03-03T06:26:58.526662+0000","last_clean":"2023-03-03T06:26:58.526662+0000","last_became_active":"2023-03-03T06:26:57.532476+0000","last_became_peered":"2023-03-03T06:26:57.532476+0000","last_unstale":"2023-03-03T06:26:58.526662+0000","last_undegraded":"2023-03-03T06:26:58.526662+0000","last_fullsized":"2023-03-03T06:26:58.526662+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:52:38.600353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.555974+0000","last_change":"2023-03-03T06:26:57.554709+0000","last_active":"2023-03-03T06:26:57.555974+0000","last_peered":"2023-03-03T06:26:57.555974+0000","last_clean":"2023-03-03T06:26:57.555974+0000","last_became_active":"2023-03-03T06:26:57.554350+0000","last_became_peered":"2023-03-03T06:26:57.554350+0000","last_unstale":"2023-03-03T06:26:57.555974+0000","last_undegraded":"2023-03-03T06:26:57.555974+0000","last_fullsized":"2023-03-03T06:26:57.555974+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:07:59.587114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.554753+0000","last_change":"2023-03-03T06:26:57.551033+0000","last_active":"2023-03-03T06:26:57.554753+0000","last_peered":"2023-03-03T06:26:57.554753+0000","last_clean":"2023-03-03T06:26:57.554753+0000","last_became_active":"2023-03-03T06:26:57.550377+0000","last_became_peered":"2023-03-03T06:26:57.550377+0000","last_unstale":"2023-03-03T06:26:57.554753+0000","last_undegraded":"2023-03-03T06:26:57.554753+0000","last_fullsized":"2023-03-03T06:26:57.554753+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:44:40.208203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.551947+0000","last_change":"2023-03-03T06:26:57.544468+0000","last_active":"2023-03-03T06:26:57.551947+0000","last_peered":"2023-03-03T06:26:57.551947+0000","last_clean":"2023-03-03T06:26:57.551947+0000","last_became_active":"2023-03-03T06:26:57.544183+0000","last_became_peered":"2023-03-03T06:26:57.544183+0000","last_unstale":"2023-03-03T06:26:57.551947+0000","last_undegraded":"2023-03-03T06:26:57.551947+0000","last_fullsized":"2023-03-03T06:26:57.551947+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:08:37.698278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.534698+0000","last_change":"2023-03-03T06:26:57.532648+0000","last_active":"2023-03-03T06:26:57.534698+0000","last_peered":"2023-03-03T06:26:57.534698+0000","last_clean":"2023-03-03T06:26:57.534698+0000","last_became_active":"2023-03-03T06:26:57.531621+0000","last_became_peered":"2023-03-03T06:26:57.531621+0000","last_unstale":"2023-03-03T06:26:57.534698+0000","last_undegraded":"2023-03-03T06:26:57.534698+0000","last_fullsized":"2023-03-03T06:26:57.534698+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:33:39.564321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.551432+0000","last_change":"2023-03-03T06:26:57.544796+0000","last_active":"2023-03-03T06:26:57.551432+0000","last_peered":"2023-03-03T06:26:57.551432+0000","last_clean":"2023-03-03T06:26:57.551432+0000","last_became_active":"2023-03-03T06:26:57.544261+0000","last_became_peered":"2023-03-03T06:26:57.544261+0000","last_unstale":"2023-03-03T06:26:57.551432+0000","last_undegraded":"2023-03-03T06:26:57.551432+0000","last_fullsized":"2023-03-03T06:26:57.551432+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:52:04.730707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.546697+0000","last_change":"2023-03-03T06:26:57.537160+0000","last_active":"2023-03-03T06:26:57.546697+0000","last_peered":"2023-03-03T06:26:57.546697+0000","last_clean":"2023-03-03T06:26:57.546697+0000","last_became_active":"2023-03-03T06:26:57.536927+0000","last_became_peered":"2023-03-03T06:26:57.536927+0000","last_unstale":"2023-03-03T06:26:57.546697+0000","last_undegraded":"2023-03-03T06:26:57.546697+0000","last_fullsized":"2023-03-03T06:26:57.546697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:52:38.600353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527620+0000","last_change":"2023-03-03T06:26:57.527963+0000","last_active":"2023-03-03T06:26:58.527620+0000","last_peered":"2023-03-03T06:26:58.527620+0000","last_clean":"2023-03-03T06:26:58.527620+0000","last_became_active":"2023-03-03T06:26:57.527354+0000","last_became_peered":"2023-03-03T06:26:57.527354+0000","last_unstale":"2023-03-03T06:26:58.527620+0000","last_undegraded":"2023-03-03T06:26:58.527620+0000","last_fullsized":"2023-03-03T06:26:58.527620+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:51:08.874774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.524933+0000","last_change":"2023-03-03T06:26:57.530138+0000","last_active":"2023-03-03T06:26:58.524933+0000","last_peered":"2023-03-03T06:26:58.524933+0000","last_clean":"2023-03-03T06:26:58.524933+0000","last_became_active":"2023-03-03T06:26:57.529800+0000","last_became_peered":"2023-03-03T06:26:57.529800+0000","last_unstale":"2023-03-03T06:26:58.524933+0000","last_undegraded":"2023-03-03T06:26:58.524933+0000","last_fullsized":"2023-03-03T06:26:58.524933+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:43:09.322094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.533606+0000","last_change":"2023-03-03T06:26:57.530356+0000","last_active":"2023-03-03T06:26:57.533606+0000","last_peered":"2023-03-03T06:26:57.533606+0000","last_clean":"2023-03-03T06:26:57.533606+0000","last_became_active":"2023-03-03T06:26:57.529492+0000","last_became_peered":"2023-03-03T06:26:57.529492+0000","last_unstale":"2023-03-03T06:26:57.533606+0000","last_undegraded":"2023-03-03T06:26:57.533606+0000","last_fullsized":"2023-03-03T06:26:57.533606+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:22:07.864707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.533230+0000","last_change":"2023-03-03T06:26:57.523517+0000","last_active":"2023-03-03T06:26:57.533230+0000","last_peered":"2023-03-03T06:26:57.533230+0000","last_clean":"2023-03-03T06:26:57.533230+0000","last_became_active":"2023-03-03T06:26:57.523256+0000","last_became_peered":"2023-03-03T06:26:57.523256+0000","last_unstale":"2023-03-03T06:26:57.533230+0000","last_undegraded":"2023-03-03T06:26:57.533230+0000","last_fullsized":"2023-03-03T06:26:57.533230+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:28:14.978083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.549335+0000","last_change":"2023-03-03T06:26:57.541331+0000","last_active":"2023-03-03T06:26:57.549335+0000","last_peered":"2023-03-03T06:26:57.549335+0000","last_clean":"2023-03-03T06:26:57.549335+0000","last_became_active":"2023-03-03T06:26:57.541053+0000","last_became_peered":"2023-03-03T06:26:57.541053+0000","last_unstale":"2023-03-03T06:26:57.549335+0000","last_undegraded":"2023-03-03T06:26:57.549335+0000","last_fullsized":"2023-03-03T06:26:57.549335+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:35:16.888845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527661+0000","last_change":"2023-03-03T06:26:57.521235+0000","last_active":"2023-03-03T06:26:58.527661+0000","last_peered":"2023-03-03T06:26:58.527661+0000","last_clean":"2023-03-03T06:26:58.527661+0000","last_became_active":"2023-03-03T06:26:57.520875+0000","last_became_peered":"2023-03-03T06:26:57.520875+0000","last_unstale":"2023-03-03T06:26:58.527661+0000","last_undegraded":"2023-03-03T06:26:58.527661+0000","last_fullsized":"2023-03-03T06:26:58.527661+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:33:32.980274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527160+0000","last_change":"2023-03-03T06:26:57.539157+0000","last_active":"2023-03-03T06:26:58.527160+0000","last_peered":"2023-03-03T06:26:58.527160+0000","last_clean":"2023-03-03T06:26:58.527160+0000","last_became_active":"2023-03-03T06:26:57.538499+0000","last_became_peered":"2023-03-03T06:26:57.538499+0000","last_unstale":"2023-03-03T06:26:58.527160+0000","last_undegraded":"2023-03-03T06:26:58.527160+0000","last_fullsized":"2023-03-03T06:26:58.527160+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:52:04.730707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.557646+0000","last_change":"2023-03-03T06:26:57.543141+0000","last_active":"2023-03-03T06:26:57.557646+0000","last_peered":"2023-03-03T06:26:57.557646+0000","last_clean":"2023-03-03T06:26:57.557646+0000","last_became_active":"2023-03-03T06:26:57.542748+0000","last_became_peered":"2023-03-03T06:26:57.542748+0000","last_unstale":"2023-03-03T06:26:57.557646+0000","last_undegraded":"2023-03-03T06:26:57.557646+0000","last_fullsized":"2023-03-03T06:26:57.557646+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:21:22.126399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.525087+0000","last_change":"2023-03-03T06:26:57.538527+0000","last_active":"2023-03-03T06:26:58.525087+0000","last_peered":"2023-03-03T06:26:58.525087+0000","last_clean":"2023-03-03T06:26:58.525087+0000","last_became_active":"2023-03-03T06:26:57.538169+0000","last_became_peered":"2023-03-03T06:26:57.538169+0000","last_unstale":"2023-03-03T06:26:58.525087+0000","last_undegraded":"2023-03-03T06:26:58.525087+0000","last_fullsized":"2023-03-03T06:26:58.525087+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:53:29.523443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.547137+0000","last_change":"2023-03-03T06:26:57.543402+0000","last_active":"2023-03-03T06:26:57.547137+0000","last_peered":"2023-03-03T06:26:57.547137+0000","last_clean":"2023-03-03T06:26:57.547137+0000","last_became_active":"2023-03-03T06:26:57.543152+0000","last_became_peered":"2023-03-03T06:26:57.543152+0000","last_unstale":"2023-03-03T06:26:57.547137+0000","last_undegraded":"2023-03-03T06:26:57.547137+0000","last_fullsized":"2023-03-03T06:26:57.547137+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:24:15.711569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.551028+0000","last_change":"2023-03-03T06:26:57.543683+0000","last_active":"2023-03-03T06:26:57.551028+0000","last_peered":"2023-03-03T06:26:57.551028+0000","last_clean":"2023-03-03T06:26:57.551028+0000","last_became_active":"2023-03-03T06:26:57.543378+0000","last_became_peered":"2023-03-03T06:26:57.543378+0000","last_unstale":"2023-03-03T06:26:57.551028+0000","last_undegraded":"2023-03-03T06:26:57.551028+0000","last_fullsized":"2023-03-03T06:26:57.551028+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:00:05.544209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.559502+0000","last_change":"2023-03-03T06:26:57.552792+0000","last_active":"2023-03-03T06:26:57.559502+0000","last_peered":"2023-03-03T06:26:57.559502+0000","last_clean":"2023-03-03T06:26:57.559502+0000","last_became_active":"2023-03-03T06:26:57.552414+0000","last_became_peered":"2023-03-03T06:26:57.552414+0000","last_unstale":"2023-03-03T06:26:57.559502+0000","last_undegraded":"2023-03-03T06:26:57.559502+0000","last_fullsized":"2023-03-03T06:26:57.559502+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:40:53.187472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.559464+0000","last_change":"2023-03-03T06:26:57.551622+0000","last_active":"2023-03-03T06:26:57.559464+0000","last_peered":"2023-03-03T06:26:57.559464+0000","last_clean":"2023-03-03T06:26:57.559464+0000","last_became_active":"2023-03-03T06:26:57.551346+0000","last_became_peered":"2023-03-03T06:26:57.551346+0000","last_unstale":"2023-03-03T06:26:57.559464+0000","last_undegraded":"2023-03-03T06:26:57.559464+0000","last_fullsized":"2023-03-03T06:26:57.559464+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:22:26.543385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.557764+0000","last_change":"2023-03-03T06:26:57.545789+0000","last_active":"2023-03-03T06:26:57.557764+0000","last_peered":"2023-03-03T06:26:57.557764+0000","last_clean":"2023-03-03T06:26:57.557764+0000","last_became_active":"2023-03-03T06:26:57.545236+0000","last_became_peered":"2023-03-03T06:26:57.545236+0000","last_unstale":"2023-03-03T06:26:57.557764+0000","last_undegraded":"2023-03-03T06:26:57.557764+0000","last_fullsized":"2023-03-03T06:26:57.557764+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:01:51.362166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.525116+0000","last_change":"2023-03-03T06:26:57.536273+0000","last_active":"2023-03-03T06:26:58.525116+0000","last_peered":"2023-03-03T06:26:58.525116+0000","last_clean":"2023-03-03T06:26:58.525116+0000","last_became_active":"2023-03-03T06:26:57.535981+0000","last_became_peered":"2023-03-03T06:26:57.535981+0000","last_unstale":"2023-03-03T06:26:58.525116+0000","last_undegraded":"2023-03-03T06:26:58.525116+0000","last_fullsized":"2023-03-03T06:26:58.525116+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:01:51.362166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526966+0000","last_change":"2023-03-03T06:26:57.525440+0000","last_active":"2023-03-03T06:26:58.526966+0000","last_peered":"2023-03-03T06:26:58.526966+0000","last_clean":"2023-03-03T06:26:58.526966+0000","last_became_active":"2023-03-03T06:26:57.525141+0000","last_became_peered":"2023-03-03T06:26:57.525141+0000","last_unstale":"2023-03-03T06:26:58.526966+0000","last_undegraded":"2023-03-03T06:26:58.526966+0000","last_fullsized":"2023-03-03T06:26:58.526966+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:22:07.864707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527302+0000","last_change":"2023-03-03T06:26:57.528132+0000","last_active":"2023-03-03T06:26:58.527302+0000","last_peered":"2023-03-03T06:26:58.527302+0000","last_clean":"2023-03-03T06:26:58.527302+0000","last_became_active":"2023-03-03T06:26:57.527678+0000","last_became_peered":"2023-03-03T06:26:57.527678+0000","last_unstale":"2023-03-03T06:26:58.527302+0000","last_undegraded":"2023-03-03T06:26:58.527302+0000","last_fullsized":"2023-03-03T06:26:58.527302+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:25:57.269311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.549166+0000","last_change":"2023-03-03T06:26:57.539824+0000","last_active":"2023-03-03T06:26:57.549166+0000","last_peered":"2023-03-03T06:26:57.549166+0000","last_clean":"2023-03-03T06:26:57.549166+0000","last_became_active":"2023-03-03T06:26:57.538206+0000","last_became_peered":"2023-03-03T06:26:57.538206+0000","last_unstale":"2023-03-03T06:26:57.549166+0000","last_undegraded":"2023-03-03T06:26:57.549166+0000","last_fullsized":"2023-03-03T06:26:57.549166+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:43:09.322094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526534+0000","last_change":"2023-03-03T06:26:57.552343+0000","last_active":"2023-03-03T06:26:58.526534+0000","last_peered":"2023-03-03T06:26:58.526534+0000","last_clean":"2023-03-03T06:26:58.526534+0000","last_became_active":"2023-03-03T06:26:57.552066+0000","last_became_peered":"2023-03-03T06:26:57.552066+0000","last_unstale":"2023-03-03T06:26:58.526534+0000","last_undegraded":"2023-03-03T06:26:58.526534+0000","last_fullsized":"2023-03-03T06:26:58.526534+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:33:39.564321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.533214+0000","last_change":"2023-03-03T06:26:57.523031+0000","last_active":"2023-03-03T06:26:57.533214+0000","last_peered":"2023-03-03T06:26:57.533214+0000","last_clean":"2023-03-03T06:26:57.533214+0000","last_became_active":"2023-03-03T06:26:57.522770+0000","last_became_peered":"2023-03-03T06:26:57.522770+0000","last_unstale":"2023-03-03T06:26:57.533214+0000","last_undegraded":"2023-03-03T06:26:57.533214+0000","last_fullsized":"2023-03-03T06:26:57.533214+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:12:36.268936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.524715+0000","last_change":"2023-03-03T06:26:57.537584+0000","last_active":"2023-03-03T06:26:58.524715+0000","last_peered":"2023-03-03T06:26:58.524715+0000","last_clean":"2023-03-03T06:26:58.524715+0000","last_became_active":"2023-03-03T06:26:57.532219+0000","last_became_peered":"2023-03-03T06:26:57.532219+0000","last_unstale":"2023-03-03T06:26:58.524715+0000","last_undegraded":"2023-03-03T06:26:58.524715+0000","last_fullsized":"2023-03-03T06:26:58.524715+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:47:58.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.535226+0000","last_change":"2023-03-03T06:26:57.528171+0000","last_active":"2023-03-03T06:26:57.535226+0000","last_peered":"2023-03-03T06:26:57.535226+0000","last_clean":"2023-03-03T06:26:57.535226+0000","last_became_active":"2023-03-03T06:26:57.527905+0000","last_became_peered":"2023-03-03T06:26:57.527905+0000","last_unstale":"2023-03-03T06:26:57.535226+0000","last_undegraded":"2023-03-03T06:26:57.535226+0000","last_fullsized":"2023-03-03T06:26:57.535226+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:18:43.324949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.559402+0000","last_change":"2023-03-03T06:26:57.009356+0000","last_active":"2023-03-03T06:26:57.559402+0000","last_peered":"2023-03-03T06:26:57.559402+0000","last_clean":"2023-03-03T06:26:57.559402+0000","last_became_active":"2023-03-03T06:26:57.009060+0000","last_became_peered":"2023-03-03T06:26:57.009060+0000","last_unstale":"2023-03-03T06:26:57.559402+0000","last_undegraded":"2023-03-03T06:26:57.559402+0000","last_fullsized":"2023-03-03T06:26:57.559402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:26:53.257291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.557743+0000","last_change":"2023-03-03T06:26:57.545558+0000","last_active":"2023-03-03T06:26:57.557743+0000","last_peered":"2023-03-03T06:26:57.557743+0000","last_clean":"2023-03-03T06:26:57.557743+0000","last_became_active":"2023-03-03T06:26:57.544882+0000","last_became_peered":"2023-03-03T06:26:57.544882+0000","last_unstale":"2023-03-03T06:26:57.557743+0000","last_undegraded":"2023-03-03T06:26:57.557743+0000","last_fullsized":"2023-03-03T06:26:57.557743+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:47:58.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527312+0000","last_change":"2023-03-03T06:26:57.010540+0000","last_active":"2023-03-03T06:26:58.527312+0000","last_peered":"2023-03-03T06:26:58.527312+0000","last_clean":"2023-03-03T06:26:58.527312+0000","last_became_active":"2023-03-03T06:26:57.010119+0000","last_became_peered":"2023-03-03T06:26:57.010119+0000","last_unstale":"2023-03-03T06:26:58.527312+0000","last_undegraded":"2023-03-03T06:26:58.527312+0000","last_fullsized":"2023-03-03T06:26:58.527312+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:29:45.742607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527459+0000","last_change":"2023-03-03T06:26:57.530333+0000","last_active":"2023-03-03T06:26:58.527459+0000","last_peered":"2023-03-03T06:26:58.527459+0000","last_clean":"2023-03-03T06:26:58.527459+0000","last_became_active":"2023-03-03T06:26:57.529967+0000","last_became_peered":"2023-03-03T06:26:57.529967+0000","last_unstale":"2023-03-03T06:26:58.527459+0000","last_undegraded":"2023-03-03T06:26:58.527459+0000","last_fullsized":"2023-03-03T06:26:58.527459+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:44:40.208203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.541580+0000","last_change":"2023-03-03T06:26:57.000595+0000","last_active":"2023-03-03T06:26:57.541580+0000","last_peered":"2023-03-03T06:26:57.541580+0000","last_clean":"2023-03-03T06:26:57.541580+0000","last_became_active":"2023-03-03T06:26:56.999153+0000","last_became_peered":"2023-03-03T06:26:56.999153+0000","last_unstale":"2023-03-03T06:26:57.541580+0000","last_undegraded":"2023-03-03T06:26:57.541580+0000","last_fullsized":"2023-03-03T06:26:57.541580+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:47:57.848563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.547908+0000","last_change":"2023-03-03T06:26:57.538595+0000","last_active":"2023-03-03T06:26:57.547908+0000","last_peered":"2023-03-03T06:26:57.547908+0000","last_clean":"2023-03-03T06:26:57.547908+0000","last_became_active":"2023-03-03T06:26:57.538271+0000","last_became_peered":"2023-03-03T06:26:57.538271+0000","last_unstale":"2023-03-03T06:26:57.547908+0000","last_undegraded":"2023-03-03T06:26:57.547908+0000","last_fullsized":"2023-03-03T06:26:57.547908+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:01:02.587632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.554107+0000","last_change":"2023-03-03T06:26:56.997835+0000","last_active":"2023-03-03T06:26:57.554107+0000","last_peered":"2023-03-03T06:26:57.554107+0000","last_clean":"2023-03-03T06:26:57.554107+0000","last_became_active":"2023-03-03T06:26:56.997498+0000","last_became_peered":"2023-03-03T06:26:56.997498+0000","last_unstale":"2023-03-03T06:26:57.554107+0000","last_undegraded":"2023-03-03T06:26:57.554107+0000","last_fullsized":"2023-03-03T06:26:57.554107+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:40:52.170576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526400+0000","last_change":"2023-03-03T06:26:57.541244+0000","last_active":"2023-03-03T06:26:58.526400+0000","last_peered":"2023-03-03T06:26:58.526400+0000","last_clean":"2023-03-03T06:26:58.526400+0000","last_became_active":"2023-03-03T06:26:57.540994+0000","last_became_peered":"2023-03-03T06:26:57.540994+0000","last_unstale":"2023-03-03T06:26:58.526400+0000","last_undegraded":"2023-03-03T06:26:58.526400+0000","last_fullsized":"2023-03-03T06:26:58.526400+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:54:41.470293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.553556+0000","last_change":"2023-03-03T06:26:56.997151+0000","last_active":"2023-03-03T06:26:57.553556+0000","last_peered":"2023-03-03T06:26:57.553556+0000","last_clean":"2023-03-03T06:26:57.553556+0000","last_became_active":"2023-03-03T06:26:56.996608+0000","last_became_peered":"2023-03-03T06:26:56.996608+0000","last_unstale":"2023-03-03T06:26:57.553556+0000","last_undegraded":"2023-03-03T06:26:57.553556+0000","last_fullsized":"2023-03-03T06:26:57.553556+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:18.385759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.531988+0000","last_change":"2023-03-03T06:26:57.528809+0000","last_active":"2023-03-03T06:26:57.531988+0000","last_peered":"2023-03-03T06:26:57.531988+0000","last_clean":"2023-03-03T06:26:57.531988+0000","last_became_active":"2023-03-03T06:26:57.528423+0000","last_became_peered":"2023-03-03T06:26:57.528423+0000","last_unstale":"2023-03-03T06:26:57.531988+0000","last_undegraded":"2023-03-03T06:26:57.531988+0000","last_fullsized":"2023-03-03T06:26:57.531988+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:11:30.592668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526522+0000","last_change":"2023-03-03T06:26:56.999645+0000","last_active":"2023-03-03T06:26:58.526522+0000","last_peered":"2023-03-03T06:26:58.526522+0000","last_clean":"2023-03-03T06:26:58.526522+0000","last_became_active":"2023-03-03T06:26:56.999366+0000","last_became_peered":"2023-03-03T06:26:56.999366+0000","last_unstale":"2023-03-03T06:26:58.526522+0000","last_undegraded":"2023-03-03T06:26:58.526522+0000","last_fullsized":"2023-03-03T06:26:58.526522+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:09:06.442642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.551929+0000","last_change":"2023-03-03T06:26:57.544366+0000","last_active":"2023-03-03T06:26:57.551929+0000","last_peered":"2023-03-03T06:26:57.551929+0000","last_clean":"2023-03-03T06:26:57.551929+0000","last_became_active":"2023-03-03T06:26:57.543811+0000","last_became_peered":"2023-03-03T06:26:57.543811+0000","last_unstale":"2023-03-03T06:26:57.551929+0000","last_undegraded":"2023-03-03T06:26:57.551929+0000","last_fullsized":"2023-03-03T06:26:57.551929+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:54:41.470293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526683+0000","last_change":"2023-03-03T06:26:56.997092+0000","last_active":"2023-03-03T06:26:58.526683+0000","last_peered":"2023-03-03T06:26:58.526683+0000","last_clean":"2023-03-03T06:26:58.526683+0000","last_became_active":"2023-03-03T06:26:56.996785+0000","last_became_peered":"2023-03-03T06:26:56.996785+0000","last_unstale":"2023-03-03T06:26:58.526683+0000","last_undegraded":"2023-03-03T06:26:58.526683+0000","last_fullsized":"2023-03-03T06:26:58.526683+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:21:21.109503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.559483+0000","last_change":"2023-03-03T06:26:57.552371+0000","last_active":"2023-03-03T06:26:57.559483+0000","last_peered":"2023-03-03T06:26:57.559483+0000","last_clean":"2023-03-03T06:26:57.559483+0000","last_became_active":"2023-03-03T06:26:57.552059+0000","last_became_peered":"2023-03-03T06:26:57.552059+0000","last_unstale":"2023-03-03T06:26:57.559483+0000","last_undegraded":"2023-03-03T06:26:57.559483+0000","last_fullsized":"2023-03-03T06:26:57.559483+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:18:56.146005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.518819+0000","last_change":"2023-03-03T06:26:57.013410+0000","last_active":"2023-03-03T06:26:57.518819+0000","last_peered":"2023-03-03T06:26:57.518819+0000","last_clean":"2023-03-03T06:26:57.518819+0000","last_became_active":"2023-03-03T06:26:57.013127+0000","last_became_peered":"2023-03-03T06:26:57.013127+0000","last_unstale":"2023-03-03T06:26:57.518819+0000","last_undegraded":"2023-03-03T06:26:57.518819+0000","last_fullsized":"2023-03-03T06:26:57.518819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:42:50.798978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527180+0000","last_change":"2023-03-03T06:26:57.539341+0000","last_active":"2023-03-03T06:26:58.527180+0000","last_peered":"2023-03-03T06:26:58.527180+0000","last_clean":"2023-03-03T06:26:58.527180+0000","last_became_active":"2023-03-03T06:26:57.538826+0000","last_became_peered":"2023-03-03T06:26:57.538826+0000","last_unstale":"2023-03-03T06:26:58.527180+0000","last_undegraded":"2023-03-03T06:26:58.527180+0000","last_fullsized":"2023-03-03T06:26:58.527180+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:33:49.811191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527529+0000","last_change":"2023-03-03T06:26:57.002111+0000","last_active":"2023-03-03T06:26:58.527529+0000","last_peered":"2023-03-03T06:26:58.527529+0000","last_clean":"2023-03-03T06:26:58.527529+0000","last_became_active":"2023-03-03T06:26:57.001827+0000","last_became_peered":"2023-03-03T06:26:57.001827+0000","last_unstale":"2023-03-03T06:26:58.527529+0000","last_undegraded":"2023-03-03T06:26:58.527529+0000","last_fullsized":"2023-03-03T06:26:58.527529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:52:37.583457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.554764+0000","last_change":"2023-03-03T06:26:57.550916+0000","last_active":"2023-03-03T06:26:57.554764+0000","last_peered":"2023-03-03T06:26:57.554764+0000","last_clean":"2023-03-03T06:26:57.554764+0000","last_became_active":"2023-03-03T06:26:57.550239+0000","last_became_peered":"2023-03-03T06:26:57.550239+0000","last_unstale":"2023-03-03T06:26:57.554764+0000","last_undegraded":"2023-03-03T06:26:57.554764+0000","last_fullsized":"2023-03-03T06:26:57.554764+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:51:08.874774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.522572+0000","last_change":"2023-03-03T06:26:57.006868+0000","last_active":"2023-03-03T06:26:57.522572+0000","last_peered":"2023-03-03T06:26:57.522572+0000","last_clean":"2023-03-03T06:26:57.522572+0000","last_became_active":"2023-03-03T06:26:57.003408+0000","last_became_peered":"2023-03-03T06:26:57.003408+0000","last_unstale":"2023-03-03T06:26:57.522572+0000","last_undegraded":"2023-03-03T06:26:57.522572+0000","last_fullsized":"2023-03-03T06:26:57.522572+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:18:55.129109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.535402+0000","last_change":"2023-03-03T06:26:57.527926+0000","last_active":"2023-03-03T06:26:57.535402+0000","last_peered":"2023-03-03T06:26:57.535402+0000","last_clean":"2023-03-03T06:26:57.535402+0000","last_became_active":"2023-03-03T06:26:57.527475+0000","last_became_peered":"2023-03-03T06:26:57.527475+0000","last_unstale":"2023-03-03T06:26:57.535402+0000","last_undegraded":"2023-03-03T06:26:57.535402+0000","last_fullsized":"2023-03-03T06:26:57.535402+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:02:20.606207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.558323+0000","last_change":"2023-03-03T06:26:57.009453+0000","last_active":"2023-03-03T06:26:57.558323+0000","last_peered":"2023-03-03T06:26:57.558323+0000","last_clean":"2023-03-03T06:26:57.558323+0000","last_became_active":"2023-03-03T06:26:57.009194+0000","last_became_peered":"2023-03-03T06:26:57.009194+0000","last_unstale":"2023-03-03T06:26:57.558323+0000","last_undegraded":"2023-03-03T06:26:57.558323+0000","last_fullsized":"2023-03-03T06:26:57.558323+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:18:42.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.552789+0000","last_change":"2023-03-03T06:26:57.548226+0000","last_active":"2023-03-03T06:26:57.552789+0000","last_peered":"2023-03-03T06:26:57.552789+0000","last_clean":"2023-03-03T06:26:57.552789+0000","last_became_active":"2023-03-03T06:26:57.547920+0000","last_became_peered":"2023-03-03T06:26:57.547920+0000","last_unstale":"2023-03-03T06:26:57.552789+0000","last_undegraded":"2023-03-03T06:26:57.552789+0000","last_fullsized":"2023-03-03T06:26:57.552789+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:26:54.274187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.525104+0000","last_change":"2023-03-03T06:26:57.006387+0000","last_active":"2023-03-03T06:26:58.525104+0000","last_peered":"2023-03-03T06:26:58.525104+0000","last_clean":"2023-03-03T06:26:58.525104+0000","last_became_active":"2023-03-03T06:26:57.006088+0000","last_became_peered":"2023-03-03T06:26:57.006088+0000","last_unstale":"2023-03-03T06:26:58.525104+0000","last_undegraded":"2023-03-03T06:26:58.525104+0000","last_fullsized":"2023-03-03T06:26:58.525104+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:27:52.947467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.557764+0000","last_change":"2023-03-03T06:26:57.545237+0000","last_active":"2023-03-03T06:26:57.557764+0000","last_peered":"2023-03-03T06:26:57.557764+0000","last_clean":"2023-03-03T06:26:57.557764+0000","last_became_active":"2023-03-03T06:26:57.544543+0000","last_became_peered":"2023-03-03T06:26:57.544543+0000","last_unstale":"2023-03-03T06:26:57.557764+0000","last_undegraded":"2023-03-03T06:26:57.557764+0000","last_fullsized":"2023-03-03T06:26:57.557764+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:42:51.815874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.553243+0000","last_change":"2023-03-03T06:26:57.005116+0000","last_active":"2023-03-03T06:26:57.553243+0000","last_peered":"2023-03-03T06:26:57.553243+0000","last_clean":"2023-03-03T06:26:57.553243+0000","last_became_active":"2023-03-03T06:26:57.004870+0000","last_became_peered":"2023-03-03T06:26:57.004870+0000","last_unstale":"2023-03-03T06:26:57.553243+0000","last_undegraded":"2023-03-03T06:26:57.553243+0000","last_fullsized":"2023-03-03T06:26:57.553243+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:34:53.416403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.555350+0000","last_change":"2023-03-03T06:26:57.551643+0000","last_active":"2023-03-03T06:26:57.555350+0000","last_peered":"2023-03-03T06:26:57.555350+0000","last_clean":"2023-03-03T06:26:57.555350+0000","last_became_active":"2023-03-03T06:26:57.551348+0000","last_became_peered":"2023-03-03T06:26:57.551348+0000","last_unstale":"2023-03-03T06:26:57.555350+0000","last_undegraded":"2023-03-03T06:26:57.555350+0000","last_fullsized":"2023-03-03T06:26:57.555350+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:33:32.980274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.537206+0000","last_change":"2023-03-03T06:26:57.002269+0000","last_active":"2023-03-03T06:26:57.537206+0000","last_peered":"2023-03-03T06:26:57.537206+0000","last_clean":"2023-03-03T06:26:57.537206+0000","last_became_active":"2023-03-03T06:26:57.001962+0000","last_became_peered":"2023-03-03T06:26:57.001962+0000","last_unstale":"2023-03-03T06:26:57.537206+0000","last_undegraded":"2023-03-03T06:26:57.537206+0000","last_fullsized":"2023-03-03T06:26:57.537206+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:35:15.871949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.533187+0000","last_change":"2023-03-03T06:26:57.527074+0000","last_active":"2023-03-03T06:26:57.533187+0000","last_peered":"2023-03-03T06:26:57.533187+0000","last_clean":"2023-03-03T06:26:57.533187+0000","last_became_active":"2023-03-03T06:26:57.526791+0000","last_became_peered":"2023-03-03T06:26:57.526791+0000","last_unstale":"2023-03-03T06:26:57.533187+0000","last_undegraded":"2023-03-03T06:26:57.533187+0000","last_fullsized":"2023-03-03T06:26:57.533187+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:16:19.516452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.513832+0000","last_change":"2023-03-03T06:26:57.012556+0000","last_active":"2023-03-03T06:26:57.513832+0000","last_peered":"2023-03-03T06:26:57.513832+0000","last_clean":"2023-03-03T06:26:57.513832+0000","last_became_active":"2023-03-03T06:26:57.012011+0000","last_became_peered":"2023-03-03T06:26:57.012011+0000","last_unstale":"2023-03-03T06:26:57.513832+0000","last_undegraded":"2023-03-03T06:26:57.513832+0000","last_fullsized":"2023-03-03T06:26:57.513832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:58:42.202828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.554775+0000","last_change":"2023-03-03T06:26:57.551237+0000","last_active":"2023-03-03T06:26:57.554775+0000","last_peered":"2023-03-03T06:26:57.554775+0000","last_clean":"2023-03-03T06:26:57.554775+0000","last_became_active":"2023-03-03T06:26:57.550693+0000","last_became_peered":"2023-03-03T06:26:57.550693+0000","last_unstale":"2023-03-03T06:26:57.554775+0000","last_undegraded":"2023-03-03T06:26:57.554775+0000","last_fullsized":"2023-03-03T06:26:57.554775+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:31:36.667958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.555442+0000","last_change":"2023-03-03T06:26:57.011832+0000","last_active":"2023-03-03T06:26:57.555442+0000","last_peered":"2023-03-03T06:26:57.555442+0000","last_clean":"2023-03-03T06:26:57.555442+0000","last_became_active":"2023-03-03T06:26:57.011552+0000","last_became_peered":"2023-03-03T06:26:57.011552+0000","last_unstale":"2023-03-03T06:26:57.555442+0000","last_undegraded":"2023-03-03T06:26:57.555442+0000","last_fullsized":"2023-03-03T06:26:57.555442+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:21:21.109503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.525204+0000","last_change":"2023-03-03T06:26:57.522979+0000","last_active":"2023-03-03T06:26:58.525204+0000","last_peered":"2023-03-03T06:26:58.525204+0000","last_clean":"2023-03-03T06:26:58.525204+0000","last_became_active":"2023-03-03T06:26:57.522737+0000","last_became_peered":"2023-03-03T06:26:57.522737+0000","last_unstale":"2023-03-03T06:26:58.525204+0000","last_undegraded":"2023-03-03T06:26:58.525204+0000","last_fullsized":"2023-03-03T06:26:58.525204+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:22:26.543385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.525285+0000","last_change":"2023-03-03T06:26:57.004509+0000","last_active":"2023-03-03T06:26:58.525285+0000","last_peered":"2023-03-03T06:26:58.525285+0000","last_clean":"2023-03-03T06:26:58.525285+0000","last_became_active":"2023-03-03T06:26:57.004197+0000","last_became_peered":"2023-03-03T06:26:57.004197+0000","last_unstale":"2023-03-03T06:26:58.525285+0000","last_undegraded":"2023-03-03T06:26:58.525285+0000","last_fullsized":"2023-03-03T06:26:58.525285+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:10:41.501137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.532548+0000","last_change":"2023-03-03T06:26:57.528425+0000","last_active":"2023-03-03T06:26:57.532548+0000","last_peered":"2023-03-03T06:26:57.532548+0000","last_clean":"2023-03-03T06:26:57.532548+0000","last_became_active":"2023-03-03T06:26:57.528057+0000","last_became_peered":"2023-03-03T06:26:57.528057+0000","last_unstale":"2023-03-03T06:26:57.532548+0000","last_undegraded":"2023-03-03T06:26:57.532548+0000","last_fullsized":"2023-03-03T06:26:57.532548+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:58:43.219725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527431+0000","last_change":"2023-03-03T06:26:56.995475+0000","last_active":"2023-03-03T06:26:58.527431+0000","last_peered":"2023-03-03T06:26:58.527431+0000","last_clean":"2023-03-03T06:26:58.527431+0000","last_became_active":"2023-03-03T06:26:56.995200+0000","last_became_peered":"2023-03-03T06:26:56.995200+0000","last_unstale":"2023-03-03T06:26:58.527431+0000","last_undegraded":"2023-03-03T06:26:58.527431+0000","last_fullsized":"2023-03-03T06:26:58.527431+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:36:16.195494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.549193+0000","last_change":"2023-03-03T06:26:57.541024+0000","last_active":"2023-03-03T06:26:57.549193+0000","last_peered":"2023-03-03T06:26:57.549193+0000","last_clean":"2023-03-03T06:26:57.549193+0000","last_became_active":"2023-03-03T06:26:57.540744+0000","last_became_peered":"2023-03-03T06:26:57.540744+0000","last_unstale":"2023-03-03T06:26:57.549193+0000","last_undegraded":"2023-03-03T06:26:57.549193+0000","last_fullsized":"2023-03-03T06:26:57.549193+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:18:43.324949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527521+0000","last_change":"2023-03-03T06:26:56.996779+0000","last_active":"2023-03-03T06:26:58.527521+0000","last_peered":"2023-03-03T06:26:58.527521+0000","last_clean":"2023-03-03T06:26:58.527521+0000","last_became_active":"2023-03-03T06:26:56.996489+0000","last_became_peered":"2023-03-03T06:26:56.996489+0000","last_unstale":"2023-03-03T06:26:58.527521+0000","last_undegraded":"2023-03-03T06:26:58.527521+0000","last_fullsized":"2023-03-03T06:26:58.527521+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:42:50.798978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.559429+0000","last_change":"2023-03-03T06:26:57.550703+0000","last_active":"2023-03-03T06:26:57.559429+0000","last_peered":"2023-03-03T06:26:57.559429+0000","last_clean":"2023-03-03T06:26:57.559429+0000","last_became_active":"2023-03-03T06:26:57.548830+0000","last_became_peered":"2023-03-03T06:26:57.548830+0000","last_unstale":"2023-03-03T06:26:57.559429+0000","last_undegraded":"2023-03-03T06:26:57.559429+0000","last_fullsized":"2023-03-03T06:26:57.559429+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:53:29.523443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.516945+0000","last_change":"2023-03-03T06:26:57.009951+0000","last_active":"2023-03-03T06:26:57.516945+0000","last_peered":"2023-03-03T06:26:57.516945+0000","last_clean":"2023-03-03T06:26:57.516945+0000","last_became_active":"2023-03-03T06:26:57.009684+0000","last_became_peered":"2023-03-03T06:26:57.009684+0000","last_unstale":"2023-03-03T06:26:57.516945+0000","last_undegraded":"2023-03-03T06:26:57.516945+0000","last_fullsized":"2023-03-03T06:26:57.516945+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:44:39.191307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.533175+0000","last_change":"2023-03-03T06:26:57.527276+0000","last_active":"2023-03-03T06:26:57.533175+0000","last_peered":"2023-03-03T06:26:57.533175+0000","last_clean":"2023-03-03T06:26:57.533175+0000","last_became_active":"2023-03-03T06:26:57.526961+0000","last_became_peered":"2023-03-03T06:26:57.526961+0000","last_unstale":"2023-03-03T06:26:57.533175+0000","last_undegraded":"2023-03-03T06:26:57.533175+0000","last_fullsized":"2023-03-03T06:26:57.533175+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:00:24.063719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.553718+0000","last_change":"2023-03-03T06:26:56.997184+0000","last_active":"2023-03-03T06:26:57.553718+0000","last_peered":"2023-03-03T06:26:57.553718+0000","last_clean":"2023-03-03T06:26:57.553718+0000","last_became_active":"2023-03-03T06:26:56.996859+0000","last_became_peered":"2023-03-03T06:26:56.996859+0000","last_unstale":"2023-03-03T06:26:57.553718+0000","last_undegraded":"2023-03-03T06:26:57.553718+0000","last_fullsized":"2023-03-03T06:26:57.553718+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:31:35.651062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.529820+0000","last_change":"2023-03-03T06:26:57.521350+0000","last_active":"2023-03-03T06:26:57.529820+0000","last_peered":"2023-03-03T06:26:57.529820+0000","last_clean":"2023-03-03T06:26:57.529820+0000","last_became_active":"2023-03-03T06:26:57.521082+0000","last_became_peered":"2023-03-03T06:26:57.521082+0000","last_unstale":"2023-03-03T06:26:57.529820+0000","last_undegraded":"2023-03-03T06:26:57.529820+0000","last_fullsized":"2023-03-03T06:26:57.529820+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:42:51.815874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526767+0000","last_change":"2023-03-03T06:26:57.010508+0000","last_active":"2023-03-03T06:26:58.526767+0000","last_peered":"2023-03-03T06:26:58.526767+0000","last_clean":"2023-03-03T06:26:58.526767+0000","last_became_active":"2023-03-03T06:26:57.010216+0000","last_became_peered":"2023-03-03T06:26:57.010216+0000","last_unstale":"2023-03-03T06:26:58.526767+0000","last_undegraded":"2023-03-03T06:26:58.526767+0000","last_fullsized":"2023-03-03T06:26:58.526767+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:40:52.170576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.524794+0000","last_change":"2023-03-03T06:26:57.551051+0000","last_active":"2023-03-03T06:26:58.524794+0000","last_peered":"2023-03-03T06:26:58.524794+0000","last_clean":"2023-03-03T06:26:58.524794+0000","last_became_active":"2023-03-03T06:26:57.550590+0000","last_became_peered":"2023-03-03T06:26:57.550590+0000","last_unstale":"2023-03-03T06:26:58.524794+0000","last_undegraded":"2023-03-03T06:26:58.524794+0000","last_fullsized":"2023-03-03T06:26:58.524794+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:18:56.146005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.519265+0000","last_change":"2023-03-03T06:26:57.011481+0000","last_active":"2023-03-03T06:26:57.519265+0000","last_peered":"2023-03-03T06:26:57.519265+0000","last_clean":"2023-03-03T06:26:57.519265+0000","last_became_active":"2023-03-03T06:26:57.011012+0000","last_became_peered":"2023-03-03T06:26:57.011012+0000","last_unstale":"2023-03-03T06:26:57.519265+0000","last_undegraded":"2023-03-03T06:26:57.519265+0000","last_fullsized":"2023-03-03T06:26:57.519265+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:40:52.170576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.546546+0000","last_change":"2023-03-03T06:26:57.536646+0000","last_active":"2023-03-03T06:26:57.546546+0000","last_peered":"2023-03-03T06:26:57.546546+0000","last_clean":"2023-03-03T06:26:57.546546+0000","last_became_active":"2023-03-03T06:26:57.536386+0000","last_became_peered":"2023-03-03T06:26:57.536386+0000","last_unstale":"2023-03-03T06:26:57.546546+0000","last_undegraded":"2023-03-03T06:26:57.546546+0000","last_fullsized":"2023-03-03T06:26:57.546546+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:58:45.161662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.534425+0000","last_change":"2023-03-03T06:26:57.003280+0000","last_active":"2023-03-03T06:26:57.534425+0000","last_peered":"2023-03-03T06:26:57.534425+0000","last_clean":"2023-03-03T06:26:57.534425+0000","last_became_active":"2023-03-03T06:26:57.002201+0000","last_became_peered":"2023-03-03T06:26:57.002201+0000","last_unstale":"2023-03-03T06:26:57.534425+0000","last_undegraded":"2023-03-03T06:26:57.534425+0000","last_fullsized":"2023-03-03T06:26:57.534425+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:01:01.570736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.533154+0000","last_change":"2023-03-03T06:26:57.524538+0000","last_active":"2023-03-03T06:26:57.533154+0000","last_peered":"2023-03-03T06:26:57.533154+0000","last_clean":"2023-03-03T06:26:57.533154+0000","last_became_active":"2023-03-03T06:26:57.524260+0000","last_became_peered":"2023-03-03T06:26:57.524260+0000","last_unstale":"2023-03-03T06:26:57.533154+0000","last_undegraded":"2023-03-03T06:26:57.533154+0000","last_fullsized":"2023-03-03T06:26:57.533154+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:21:27.992654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.536393+0000","last_change":"2023-03-03T06:26:57.007428+0000","last_active":"2023-03-03T06:26:57.536393+0000","last_peered":"2023-03-03T06:26:57.536393+0000","last_clean":"2023-03-03T06:26:57.536393+0000","last_became_active":"2023-03-03T06:26:57.007065+0000","last_became_peered":"2023-03-03T06:26:57.007065+0000","last_unstale":"2023-03-03T06:26:57.536393+0000","last_undegraded":"2023-03-03T06:26:57.536393+0000","last_fullsized":"2023-03-03T06:26:57.536393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:42:50.798978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.003331+0000","last_change":"2023-03-03T06:25:07.205700+0000","last_active":"2023-03-03T06:26:57.003331+0000","last_peered":"2023-03-03T06:26:57.003331+0000","last_clean":"2023-03-03T06:26:57.003331+0000","last_became_active":"2023-03-03T06:25:07.205269+0000","last_became_peered":"2023-03-03T06:25:07.205269+0000","last_unstale":"2023-03-03T06:26:57.003331+0000","last_undegraded":"2023-03-03T06:26:57.003331+0000","last_fullsized":"2023-03-03T06:26:57.003331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:17:53.442570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.537293+0000","last_change":"2023-03-03T06:26:57.531174+0000","last_active":"2023-03-03T06:26:57.537293+0000","last_peered":"2023-03-03T06:26:57.537293+0000","last_clean":"2023-03-03T06:26:57.537293+0000","last_became_active":"2023-03-03T06:26:57.530872+0000","last_became_peered":"2023-03-03T06:26:57.530872+0000","last_unstale":"2023-03-03T06:26:57.537293+0000","last_undegraded":"2023-03-03T06:26:57.537293+0000","last_fullsized":"2023-03-03T06:26:57.537293+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:42:05.211539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526625+0000","last_change":"2023-03-03T06:26:56.997545+0000","last_active":"2023-03-03T06:26:58.526625+0000","last_peered":"2023-03-03T06:26:58.526625+0000","last_clean":"2023-03-03T06:26:58.526625+0000","last_became_active":"2023-03-03T06:26:56.997258+0000","last_became_peered":"2023-03-03T06:26:56.997258+0000","last_unstale":"2023-03-03T06:26:58.526625+0000","last_undegraded":"2023-03-03T06:26:58.526625+0000","last_fullsized":"2023-03-03T06:26:58.526625+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:23:12.695056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-03T06:26:56.990738+0000","last_change":"2023-03-03T06:25:07.204903+0000","last_active":"2023-03-03T06:26:56.990738+0000","last_peered":"2023-03-03T06:26:56.990738+0000","last_clean":"2023-03-03T06:26:56.990738+0000","last_became_active":"2023-03-03T06:25:07.204601+0000","last_became_peered":"2023-03-03T06:25:07.204601+0000","last_unstale":"2023-03-03T06:26:56.990738+0000","last_undegraded":"2023-03-03T06:26:56.990738+0000","last_fullsized":"2023-03-03T06:26:56.990738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:18:36.947912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526861+0000","last_change":"2023-03-03T06:26:57.529626+0000","last_active":"2023-03-03T06:26:58.526861+0000","last_peered":"2023-03-03T06:26:58.526861+0000","last_clean":"2023-03-03T06:26:58.526861+0000","last_became_active":"2023-03-03T06:26:57.529015+0000","last_became_peered":"2023-03-03T06:26:57.529015+0000","last_unstale":"2023-03-03T06:26:58.526861+0000","last_undegraded":"2023-03-03T06:26:58.526861+0000","last_fullsized":"2023-03-03T06:26:58.526861+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:29:46.759504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.510527+0000","last_change":"2023-03-03T06:26:57.010209+0000","last_active":"2023-03-03T06:26:57.510527+0000","last_peered":"2023-03-03T06:26:57.510527+0000","last_clean":"2023-03-03T06:26:57.510527+0000","last_became_active":"2023-03-03T06:26:57.009881+0000","last_became_peered":"2023-03-03T06:26:57.009881+0000","last_unstale":"2023-03-03T06:26:57.510527+0000","last_undegraded":"2023-03-03T06:26:57.510527+0000","last_fullsized":"2023-03-03T06:26:57.510527+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:43:08.305198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-03T06:26:56.990475+0000","last_change":"2023-03-03T06:25:07.208473+0000","last_active":"2023-03-03T06:26:56.990475+0000","last_peered":"2023-03-03T06:26:56.990475+0000","last_clean":"2023-03-03T06:26:56.990475+0000","last_became_active":"2023-03-03T06:25:07.208041+0000","last_became_peered":"2023-03-03T06:25:07.208041+0000","last_unstale":"2023-03-03T06:26:56.990475+0000","last_undegraded":"2023-03-03T06:26:56.990475+0000","last_fullsized":"2023-03-03T06:26:56.990475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:22:19.852493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526375+0000","last_change":"2023-03-03T06:26:57.530454+0000","last_active":"2023-03-03T06:26:58.526375+0000","last_peered":"2023-03-03T06:26:58.526375+0000","last_clean":"2023-03-03T06:26:58.526375+0000","last_became_active":"2023-03-03T06:26:57.530101+0000","last_became_peered":"2023-03-03T06:26:57.530101+0000","last_unstale":"2023-03-03T06:26:58.526375+0000","last_undegraded":"2023-03-03T06:26:58.526375+0000","last_fullsized":"2023-03-03T06:26:58.526375+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:38:20.227293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.518909+0000","last_change":"2023-03-03T06:26:57.011360+0000","last_active":"2023-03-03T06:26:57.518909+0000","last_peered":"2023-03-03T06:26:57.518909+0000","last_clean":"2023-03-03T06:26:57.518909+0000","last_became_active":"2023-03-03T06:26:57.010981+0000","last_became_peered":"2023-03-03T06:26:57.010981+0000","last_unstale":"2023-03-03T06:26:57.518909+0000","last_undegraded":"2023-03-03T06:26:57.518909+0000","last_fullsized":"2023-03-03T06:26:57.518909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:22:25.526489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-03T06:26:56.991460+0000","last_change":"2023-03-03T06:25:07.201865+0000","last_active":"2023-03-03T06:26:56.991460+0000","last_peered":"2023-03-03T06:26:56.991460+0000","last_clean":"2023-03-03T06:26:56.991460+0000","last_became_active":"2023-03-03T06:25:07.201411+0000","last_became_peered":"2023-03-03T06:25:07.201411+0000","last_unstale":"2023-03-03T06:26:56.991460+0000","last_undegraded":"2023-03-03T06:26:56.991460+0000","last_fullsized":"2023-03-03T06:26:56.991460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:17:53.442570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.557449+0000","last_change":"2023-03-03T06:26:57.543371+0000","last_active":"2023-03-03T06:26:57.557449+0000","last_peered":"2023-03-03T06:26:57.557449+0000","last_clean":"2023-03-03T06:26:57.557449+0000","last_became_active":"2023-03-03T06:26:57.543067+0000","last_became_peered":"2023-03-03T06:26:57.543067+0000","last_unstale":"2023-03-03T06:26:57.557449+0000","last_undegraded":"2023-03-03T06:26:57.557449+0000","last_fullsized":"2023-03-03T06:26:57.557449+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:01:02.587632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.515339+0000","last_change":"2023-03-03T06:26:57.001058+0000","last_active":"2023-03-03T06:26:57.515339+0000","last_peered":"2023-03-03T06:26:57.515339+0000","last_clean":"2023-03-03T06:26:57.515339+0000","last_became_active":"2023-03-03T06:26:57.000798+0000","last_became_peered":"2023-03-03T06:26:57.000798+0000","last_unstale":"2023-03-03T06:26:57.515339+0000","last_undegraded":"2023-03-03T06:26:57.515339+0000","last_fullsized":"2023-03-03T06:26:57.515339+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:25:56.252415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.513829+0000","last_change":"2023-03-03T06:25:07.201941+0000","last_active":"2023-03-03T06:26:57.513829+0000","last_peered":"2023-03-03T06:26:57.513829+0000","last_clean":"2023-03-03T06:26:57.513829+0000","last_became_active":"2023-03-03T06:25:07.201437+0000","last_became_peered":"2023-03-03T06:25:07.201437+0000","last_unstale":"2023-03-03T06:26:57.513829+0000","last_undegraded":"2023-03-03T06:26:57.513829+0000","last_fullsized":"2023-03-03T06:26:57.513829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:48:52.145810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.543721+0000","last_change":"2023-03-03T06:26:57.534202+0000","last_active":"2023-03-03T06:26:57.543721+0000","last_peered":"2023-03-03T06:26:57.543721+0000","last_clean":"2023-03-03T06:26:57.543721+0000","last_became_active":"2023-03-03T06:26:57.533864+0000","last_became_peered":"2023-03-03T06:26:57.533864+0000","last_unstale":"2023-03-03T06:26:57.543721+0000","last_undegraded":"2023-03-03T06:26:57.543721+0000","last_fullsized":"2023-03-03T06:26:57.543721+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:38:20.227293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.531866+0000","last_change":"2023-03-03T06:26:57.012078+0000","last_active":"2023-03-03T06:26:57.531866+0000","last_peered":"2023-03-03T06:26:57.531866+0000","last_clean":"2023-03-03T06:26:57.531866+0000","last_became_active":"2023-03-03T06:26:57.011761+0000","last_became_peered":"2023-03-03T06:26:57.011761+0000","last_unstale":"2023-03-03T06:26:57.531866+0000","last_undegraded":"2023-03-03T06:26:57.531866+0000","last_fullsized":"2023-03-03T06:26:57.531866+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:21:21.109503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.514710+0000","last_change":"2023-03-03T06:25:07.437275+0000","last_active":"2023-03-03T06:26:57.514710+0000","last_peered":"2023-03-03T06:26:57.514710+0000","last_clean":"2023-03-03T06:26:57.514710+0000","last_became_active":"2023-03-03T06:25:07.436823+0000","last_became_peered":"2023-03-03T06:25:07.436823+0000","last_unstale":"2023-03-03T06:26:57.514710+0000","last_undegraded":"2023-03-03T06:26:57.514710+0000","last_fullsized":"2023-03-03T06:26:57.514710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:24:15.529041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.555931+0000","last_change":"2023-03-03T06:26:57.552842+0000","last_active":"2023-03-03T06:26:57.555931+0000","last_peered":"2023-03-03T06:26:57.555931+0000","last_clean":"2023-03-03T06:26:57.555931+0000","last_became_active":"2023-03-03T06:26:57.552107+0000","last_became_peered":"2023-03-03T06:26:57.552107+0000","last_unstale":"2023-03-03T06:26:57.555931+0000","last_undegraded":"2023-03-03T06:26:57.555931+0000","last_fullsized":"2023-03-03T06:26:57.555931+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:34:54.433299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.511130+0000","last_change":"2023-03-03T06:26:57.005618+0000","last_active":"2023-03-03T06:26:57.511130+0000","last_peered":"2023-03-03T06:26:57.511130+0000","last_clean":"2023-03-03T06:26:57.511130+0000","last_became_active":"2023-03-03T06:26:57.005211+0000","last_became_peered":"2023-03-03T06:26:57.005211+0000","last_unstale":"2023-03-03T06:26:57.511130+0000","last_undegraded":"2023-03-03T06:26:57.511130+0000","last_fullsized":"2023-03-03T06:26:57.511130+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:53:28.506546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-03T06:26:56.983616+0000","last_change":"2023-03-03T06:25:07.208585+0000","last_active":"2023-03-03T06:26:56.983616+0000","last_peered":"2023-03-03T06:26:56.983616+0000","last_clean":"2023-03-03T06:26:56.983616+0000","last_became_active":"2023-03-03T06:25:07.208257+0000","last_became_peered":"2023-03-03T06:25:07.208257+0000","last_unstale":"2023-03-03T06:26:56.983616+0000","last_undegraded":"2023-03-03T06:26:56.983616+0000","last_fullsized":"2023-03-03T06:26:56.983616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:11:52.893733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.555324+0000","last_change":"2023-03-03T06:26:57.551543+0000","last_active":"2023-03-03T06:26:57.555324+0000","last_peered":"2023-03-03T06:26:57.555324+0000","last_clean":"2023-03-03T06:26:57.555324+0000","last_became_active":"2023-03-03T06:26:57.551250+0000","last_became_peered":"2023-03-03T06:26:57.551250+0000","last_unstale":"2023-03-03T06:26:57.555324+0000","last_undegraded":"2023-03-03T06:26:57.555324+0000","last_fullsized":"2023-03-03T06:26:57.555324+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:26:54.274187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527623+0000","last_change":"2023-03-03T06:26:57.002442+0000","last_active":"2023-03-03T06:26:58.527623+0000","last_peered":"2023-03-03T06:26:58.527623+0000","last_clean":"2023-03-03T06:26:58.527623+0000","last_became_active":"2023-03-03T06:26:57.002158+0000","last_became_peered":"2023-03-03T06:26:57.002158+0000","last_unstale":"2023-03-03T06:26:58.527623+0000","last_undegraded":"2023-03-03T06:26:58.527623+0000","last_fullsized":"2023-03-03T06:26:58.527623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:18:55.129109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.517767+0000","last_change":"2023-03-03T06:25:07.203115+0000","last_active":"2023-03-03T06:26:57.517767+0000","last_peered":"2023-03-03T06:26:57.517767+0000","last_clean":"2023-03-03T06:26:57.517767+0000","last_became_active":"2023-03-03T06:25:07.202705+0000","last_became_peered":"2023-03-03T06:25:07.202705+0000","last_unstale":"2023-03-03T06:26:57.517767+0000","last_undegraded":"2023-03-03T06:26:57.517767+0000","last_fullsized":"2023-03-03T06:26:57.517767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:11:32.938698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":183,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-03T06:26:56.984455+0000","last_change":"2023-03-03T06:25:02.138688+0000","last_active":"2023-03-03T06:26:56.984455+0000","last_peered":"2023-03-03T06:26:56.984455+0000","last_clean":"2023-03-03T06:26:56.984455+0000","last_became_active":"2023-03-03T06:25:02.137945+0000","last_became_peered":"2023-03-03T06:25:02.137945+0000","last_unstale":"2023-03-03T06:26:56.984455+0000","last_undegraded":"2023-03-03T06:26:56.984455+0000","last_fullsized":"2023-03-03T06:26:56.984455+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:00.679233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:00.679233+0000","last_clean_scrub_stamp":"2023-03-03T06:25:00.679233+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:24:46.304379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13932,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94357908,"statfs":{"total":96636764160,"available":96622497792,"internally_reserved":0,"allocated":1085440,"data_stored":806932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.81999999999999995}]},{"osd":3,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":3.4100000000000001,"5min":3.4100000000000001,"15min":3.4100000000000001},"last":0.91800000000000004}]},{"osd":4,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.9649999999999999,"5min":2.9649999999999999,"15min":2.9649999999999999},"last":0.88}]},{"osd":5,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":3.476,"5min":3.476,"15min":3.476},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.764,"5min":1.764,"15min":1.764},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.90600000000000003}]},{"osd":6,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":4.6139999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"min":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"max":{"1min":4.5499999999999998,"5min":4.5499999999999998,"15min":4.5499999999999998},"last":0.99399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12780,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":12288,"kb_avail":94359060,"statfs":{"total":96636764160,"available":96623677440,"internally_reserved":0,"allocated":495616,"data_stored":217130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12582912},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.21},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.819}]},{"osd":1,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.423},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"min":{"1min":1.353,"5min":1.353,"15min":1.353},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.6819999999999999}]},{"osd":2,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":1.012,"5min":1.012,"15min":1.012},"max":{"1min":2.5510000000000002,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.6379999999999999}]},{"osd":3,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":1.194,"5min":1.194,"15min":1.194},"max":{"1min":3.8559999999999999,"5min":3.8559999999999999,"15min":3.8559999999999999},"last":1.474},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"min":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.7190000000000001}]},{"osd":4,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"min":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"max":{"1min":6.4370000000000003,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":1.3100000000000001}]},{"osd":5,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.724,"15min":1.724},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":3.5059999999999998,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":1.238},{"interface":"front","average":{"1min":1.702,"5min":1.702,"15min":1.702},"min":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"max":{"1min":2.637,"5min":2.637,"15min":2.637},"last":1.532}]},{"osd":7,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"min":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"max":{"1min":3.6760000000000002,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":1.768},{"interface":"front","average":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"min":{"1min":1.419,"5min":1.419,"15min":1.419},"max":{"1min":3.4940000000000002,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":1.5940000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12788,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":12288,"kb_avail":94359052,"statfs":{"total":96636764160,"available":96623669248,"internally_reserved":0,"allocated":503808,"data_stored":223517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12582912},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.22}]},{"osd":1,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.1220000000000001}]},{"osd":2,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.47},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.288}]},{"osd":3,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"min":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.802,"5min":1.802,"15min":1.802},"min":{"1min":1.194,"5min":1.194,"15min":1.194},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.589}]},{"osd":4,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"max":{"1min":2.4350000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.407},{"interface":"front","average":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"min":{"1min":1.157,"5min":1.157,"15min":1.157},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.7509999999999999}]},{"osd":6,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"min":{"1min":1.373,"5min":1.373,"15min":1.373},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.6779999999999999},{"interface":"front","average":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"min":{"1min":1.29,"5min":1.29,"15min":1.29},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":1.706}]},{"osd":7,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.871,"15min":1.871},"min":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"max":{"1min":2.6539999999999999,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":1.5549999999999999},{"interface":"front","average":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"min":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.8839999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12988,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":12480,"kb_avail":94358852,"statfs":{"total":96636764160,"available":96623464448,"internally_reserved":0,"allocated":512000,"data_stored":224995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12779520},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":3.887,"5min":3.887,"15min":3.887},"last":1.349},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.319}]},{"osd":1,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.599},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":3.984,"5min":3.984,"15min":3.984},"last":1.105}]},{"osd":2,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.655,"5min":1.655,"15min":1.655},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":3.7669999999999999,"5min":3.7669999999999999,"15min":3.7669999999999999},"last":1.232}]},{"osd":3,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.2889999999999999}]},{"osd":5,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":1.881,"15min":1.881},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":7.0190000000000001,"5min":7.0190000000000001,"15min":7.0190000000000001},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"min":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":1.4219999999999999}]},{"osd":6,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.944,"5min":1.944,"15min":1.944},"min":{"1min":1.012,"5min":1.012,"15min":1.012},"max":{"1min":6.8810000000000002,"5min":6.8810000000000002,"15min":6.8810000000000002},"last":1.4590000000000001}]},{"osd":7,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"min":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"max":{"1min":3.5910000000000002,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.512},{"interface":"front","average":{"1min":1.748,"5min":1.748,"15min":1.748},"min":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.49}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12848,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358992,"statfs":{"total":96636764160,"available":96623607808,"internally_reserved":0,"allocated":499712,"data_stored":217096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.7949999999999999,"5min":2.7949999999999999,"15min":2.7949999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.4670000000000001,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":0.88100000000000001}]},{"osd":1,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.94899999999999995}]},{"osd":2,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.014},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":1.3600000000000001}]},{"osd":4,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":1.71},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":1.5369999999999999}]},{"osd":5,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.708,"5min":1.708,"15min":1.708},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":3.3999999999999999,"5min":3.3999999999999999,"15min":3.3999999999999999},"last":1.425},{"interface":"front","average":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"min":{"1min":1.333,"5min":1.333,"15min":1.333},"max":{"1min":2.9129999999999998,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":1.4750000000000001}]},{"osd":6,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.823,"5min":1.823,"15min":1.823},"min":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"max":{"1min":2.8860000000000001,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.9670000000000001,"5min":2.9670000000000001,"15min":2.9670000000000001},"last":2.0630000000000002}]},{"osd":7,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.964},{"interface":"front","average":{"1min":1.722,"5min":1.722,"15min":1.722},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":3.3490000000000002,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.756}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8640,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363200,"statfs":{"total":96636764160,"available":96627916800,"internally_reserved":0,"allocated":516096,"data_stored":225017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.125}]},{"osd":1,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.427},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.94299999999999995}]},{"osd":3,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.28},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.224}]},{"osd":4,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.462},{"interface":"front","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":1.504}]},{"osd":5,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"min":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"max":{"1min":3.101,"5min":3.101,"15min":3.101},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.879,"5min":1.879,"15min":1.879},"min":{"1min":1.268,"5min":1.268,"15min":1.268},"max":{"1min":2.9100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":1.6950000000000001}]},{"osd":6,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"min":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.78},{"interface":"front","average":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.925}]},{"osd":7,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"max":{"1min":3.0379999999999998,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":1.8979999999999999},{"interface":"front","average":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.829}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8744,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363096,"statfs":{"total":96636764160,"available":96627810304,"internally_reserved":0,"allocated":491520,"data_stored":216544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.879}]},{"osd":2,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.038},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":1.3340000000000001}]},{"osd":3,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":1.29}]},{"osd":4,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.383},{"interface":"front","average":{"1min":1.762,"5min":1.762,"15min":1.762},"min":{"1min":1.149,"5min":1.149,"15min":1.149},"max":{"1min":2.9399999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.5820000000000001}]},{"osd":5,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"min":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.641,"5min":1.641,"15min":1.641},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.6279999999999999}]},{"osd":6,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.764,"5min":1.764,"15min":1.764},"min":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.74}]},{"osd":7,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.845,"5min":1.845,"15min":1.845},"min":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.798,"5min":1.798,"15min":1.798},"min":{"1min":1.306,"5min":1.306,"15min":1.306},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.427}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13932,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94357908,"statfs":{"total":96636764160,"available":96622497792,"internally_reserved":0,"allocated":1085440,"data_stored":806876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.073},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.1359999999999999}]},{"osd":3,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.018},{"interface":"front","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.2190000000000001}]},{"osd":4,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":4.3419999999999996,"5min":4.3419999999999996,"15min":4.3419999999999996},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.766,"5min":1.766,"15min":1.766},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":3.0430000000000001,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":1.651}]},{"osd":5,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":1.333},{"interface":"front","average":{"1min":1.841,"5min":1.841,"15min":1.841},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":4.415,"5min":4.415,"15min":4.415},"last":1.2829999999999999}]},{"osd":6,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":3.1899999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":6.3159999999999998,"5min":6.3159999999999998,"15min":6.3159999999999998},"last":1.498}]},{"osd":7,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":5.8440000000000003,"5min":5.8440000000000003,"15min":5.8440000000000003},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.994,"5min":1.994,"15min":1.994},"min":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.617}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-03T06:27:02.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-03T06:27:02.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.726+0000 7ff0ce2ee700 1 Processor -- start 2023-03-03T06:27:02.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.726+0000 7ff0ce2ee700 1 -- start start 2023-03-03T06:27:02.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.730+0000 7ff0ce2ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0c8160880 0x7ff0c80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:02.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.730+0000 7ff0ce2ee700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0c80adec0 0x7ff0c80ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:02.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.730+0000 7ff0ce2ee700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c80ce190 con 0x7ff0c80aea70 2023-03-03T06:27:02.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.730+0000 7ff0ce2ee700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c80c99d0 con 0x7ff0c80adec0 2023-03-03T06:27:02.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.730+0000 7ff0ce2ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80cd580 con 0x7ff0c8160880 2023-03-03T06:27:02.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.730+0000 7ff0ccaeb700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff0c80aea70 0x7ff0c80ad7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51702/0 (socket says 172.21.15.3:51702) 2023-03-03T06:27:02.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.730+0000 7ff0ccaeb700 1 -- 172.21.15.3:0/2614460144 learned_addr learned my addr 172.21.15.3:0/2614460144 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:02.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.730+0000 7ff0cd2ec700 1 --2- 172.21.15.3:0/2614460144 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0c8160880 0x7ff0c80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:02.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.730+0000 7ff0cdaed700 1 --2- 172.21.15.3:0/2614460144 >> v2:172.21.15.32:3300/0 conn(0x7ff0c80adec0 0x7ff0c80ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:02.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.730+0000 7ff0bffff700 1 -- 172.21.15.3:0/2614460144 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436050117 0 0) 0x7ff0c80ce190 con 0x7ff0c80aea70 2023-03-03T06:27:02.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.730+0000 7ff0bffff700 1 -- 172.21.15.3:0/2614460144 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0b40035d0 con 0x7ff0c80aea70 2023-03-03T06:27:02.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0cd2ec700 1 -- 172.21.15.3:0/2614460144 >> v2:172.21.15.32:3300/0 conn(0x7ff0c80adec0 msgr2=0x7ff0c80ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:02.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0cd2ec700 1 --2- 172.21.15.3:0/2614460144 >> v2:172.21.15.32:3300/0 conn(0x7ff0c80adec0 0x7ff0c80ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0bffff700 1 -- 172.21.15.3:0/2614460144 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2367001542 0 0) 0x7ff0b40035d0 con 0x7ff0c80aea70 2023-03-03T06:27:02.738 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0cd2ec700 1 -- 172.21.15.3:0/2614460144 >> v1:172.21.15.3:6789/0 conn(0x7ff0c80aea70 legacy=0x7ff0c80ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:02.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0cd2ec700 1 -- 172.21.15.3:0/2614460144 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c80cd860 con 0x7ff0c8160880 2023-03-03T06:27:02.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0cd2ec700 1 --2- 172.21.15.3:0/2614460144 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0c8160880 0x7ff0c80c9dc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009850 tx=0x7ff0c4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8e2b53a14486f50 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:02.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0bffff700 1 -- 172.21.15.3:0/2614460144 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8001670 con 0x7ff0c8160880 2023-03-03T06:27:02.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0bffff700 1 -- 172.21.15.3:0/2614460144 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c4004030 con 0x7ff0c8160880 2023-03-03T06:27:02.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0bffff700 1 -- 172.21.15.3:0/2614460144 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40043e0 con 0x7ff0c8160880 2023-03-03T06:27:02.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2614460144 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0c8160880 msgr2=0x7ff0c80c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:02.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0ce2ee700 1 --2- 172.21.15.3:0/2614460144 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0c8160880 0x7ff0c80c9dc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009850 tx=0x7ff0c4009b60 comp rx=0 tx=0).stop 2023-03-03T06:27:02.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2614460144 shutdown_connections 2023-03-03T06:27:02.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0ce2ee700 1 --2- 172.21.15.3:0/2614460144 >> v2:172.21.15.32:3300/0 conn(0x7ff0c80adec0 0x7ff0c80ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0ce2ee700 1 --2- 172.21.15.3:0/2614460144 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0c8160880 0x7ff0c80c9dc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.741 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.734+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2614460144 >> 172.21.15.3:0/2614460144 conn(0x7ff0c8070a30 msgr2=0x7ff0c8070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:02.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.738+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2614460144 shutdown_connections 2023-03-03T06:27:02.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.738+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2614460144 wait complete. 2023-03-03T06:27:02.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.738+0000 7ff0ce2ee700 1 Processor -- start 2023-03-03T06:27:02.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0ce2ee700 1 -- start start 2023-03-03T06:27:02.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0ce2ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c80adec0 0x7ff0c8153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:02.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0ce2ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0c80aea70 0x7ff0c80c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:02.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0ce2ee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0c8160880 0x7ff0c80b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:02.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0ce2ee700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c8153a40 con 0x7ff0c8160880 2023-03-03T06:27:02.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0ce2ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80c6ad0 con 0x7ff0c80adec0 2023-03-03T06:27:02.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0ce2ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80c6db0 con 0x7ff0c80aea70 2023-03-03T06:27:02.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0cdaed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0c8160880 0x7ff0c80b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:02.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0cdaed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0c8160880 0x7ff0c80b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33672/0 (socket says 172.21.15.3:33672) 2023-03-03T06:27:02.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0cdaed700 1 -- 172.21.15.3:0/2021676083 learned_addr learned my addr 172.21.15.3:0/2021676083 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:02.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0ccaeb700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0c80aea70 0x7ff0c80c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:02.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0cd2ec700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c80adec0 0x7ff0c8153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:02.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0cdaed700 1 -- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0c80aea70 msgr2=0x7ff0c80c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:02.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0cdaed700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0c80aea70 0x7ff0c80c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0cdaed700 1 -- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c80adec0 msgr2=0x7ff0c8153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:02.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0cdaed700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c80adec0 0x7ff0c8153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0cdaed700 1 -- 172.21.15.3:0/2021676083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c40094b0 con 0x7ff0c8160880 2023-03-03T06:27:02.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0cd2ec700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c80adec0 0x7ff0c8153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:27:02.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.742+0000 7ff0cdaed700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0c8160880 0x7ff0c80b3730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000efc0 tx=0x7ff0c000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:02.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.746+0000 7ff0bdffb700 1 -- 172.21.15.3:0/2021676083 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c000cb80 con 0x7ff0c8160880 2023-03-03T06:27:02.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.746+0000 7ff0bdffb700 1 -- 172.21.15.3:0/2021676083 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c000f040 con 0x7ff0c8160880 2023-03-03T06:27:02.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.746+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2021676083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c80cde20 con 0x7ff0c8160880 2023-03-03T06:27:02.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.746+0000 7ff0bdffb700 1 -- 172.21.15.3:0/2021676083 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0014480 con 0x7ff0c8160880 2023-03-03T06:27:02.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.746+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2021676083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c80cdb40 con 0x7ff0c8160880 2023-03-03T06:27:02.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.746+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2021676083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c80ca300 con 0x7ff0c8160880 2023-03-03T06:27:02.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.750+0000 7ff0bdffb700 1 -- 172.21.15.3:0/2021676083 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff0c00145e0 con 0x7ff0c8160880 2023-03-03T06:27:02.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.750+0000 7ff0bdffb700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff0ac064ee0 0x7ff0ac067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:02.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.750+0000 7ff0bdffb700 1 -- 172.21.15.3:0/2021676083 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff0c00829d0 con 0x7ff0c8160880 2023-03-03T06:27:02.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.754+0000 7ff0cd2ec700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff0ac064ee0 0x7ff0ac067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:02.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.754+0000 7ff0cd2ec700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff0ac064ee0 0x7ff0ac067390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40090b0 tx=0x7ff0c4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:02.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.758+0000 7ff0bdffb700 1 -- 172.21.15.3:0/2021676083 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff0c0050940 con 0x7ff0c8160880 2023-03-03T06:27:02.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.934+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2021676083 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff0c8073e90 con 0x7ff0ac064ee0 2023-03-03T06:27:02.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.950+0000 7ff0bdffb700 1 -- 172.21.15.3:0/2021676083 <== mgr.4105 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308653 (secure 0 0 0) 0x7ff0c8073e90 con 0x7ff0ac064ee0 2023-03-03T06:27:02.959 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:27:02.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.958+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff0ac064ee0 msgr2=0x7ff0ac067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:02.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.958+0000 7ff0ce2ee700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff0ac064ee0 0x7ff0ac067390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40090b0 tx=0x7ff0c4008030 comp rx=0 tx=0).stop 2023-03-03T06:27:02.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.958+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2021676083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0c8160880 msgr2=0x7ff0c80b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:02.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.958+0000 7ff0ce2ee700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0c8160880 0x7ff0c80b3730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000efc0 tx=0x7ff0c000c0c0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.958+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2021676083 shutdown_connections 2023-03-03T06:27:02.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.958+0000 7ff0ce2ee700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7ff0ac064ee0 0x7ff0ac067390 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.958+0000 7ff0ce2ee700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0c8160880 0x7ff0c80b3730 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.958+0000 7ff0ce2ee700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7ff0c80aea70 0x7ff0c80c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.958+0000 7ff0ce2ee700 1 --2- 172.21.15.3:0/2021676083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c80adec0 0x7ff0c8153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:02.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.958+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2021676083 >> 172.21.15.3:0/2021676083 conn(0x7ff0c8070a30 msgr2=0x7ff0c80ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:02.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.958+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2021676083 shutdown_connections 2023-03-03T06:27:02.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:02.958+0000 7ff0ce2ee700 1 -- 172.21.15.3:0/2021676083 wait complete. 2023-03-03T06:27:02.968 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2023-03-03T06:27:02.997 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2023-03-03T06:27:02.439998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":96652,"kb_used_data":5068,"kb_used_omap":0,"kb_used_meta":91520,"kb_avail":754878068,"statfs":{"total":773094113280,"available":772995141632,"internally_reserved":0,"allocated":5189632,"data_stored":2938107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":93716480},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002161"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.554743+0000","last_change":"2023-03-03T06:26:57.550461+0000","last_active":"2023-03-03T06:26:57.554743+0000","last_peered":"2023-03-03T06:26:57.554743+0000","last_clean":"2023-03-03T06:26:57.554743+0000","last_became_active":"2023-03-03T06:26:57.550107+0000","last_became_peered":"2023-03-03T06:26:57.550107+0000","last_unstale":"2023-03-03T06:26:57.554743+0000","last_undegraded":"2023-03-03T06:26:57.554743+0000","last_fullsized":"2023-03-03T06:26:57.554743+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:25:57.269311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527519+0000","last_change":"2023-03-03T06:26:57.532776+0000","last_active":"2023-03-03T06:26:58.527519+0000","last_peered":"2023-03-03T06:26:58.527519+0000","last_clean":"2023-03-03T06:26:58.527519+0000","last_became_active":"2023-03-03T06:26:57.532061+0000","last_became_peered":"2023-03-03T06:26:57.532061+0000","last_unstale":"2023-03-03T06:26:58.527519+0000","last_undegraded":"2023-03-03T06:26:58.527519+0000","last_fullsized":"2023-03-03T06:26:58.527519+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:36:17.212390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526886+0000","last_change":"2023-03-03T06:26:57.530969+0000","last_active":"2023-03-03T06:26:58.526886+0000","last_peered":"2023-03-03T06:26:58.526886+0000","last_clean":"2023-03-03T06:26:58.526886+0000","last_became_active":"2023-03-03T06:26:57.530694+0000","last_became_peered":"2023-03-03T06:26:57.530694+0000","last_unstale":"2023-03-03T06:26:58.526886+0000","last_undegraded":"2023-03-03T06:26:58.526886+0000","last_fullsized":"2023-03-03T06:26:58.526886+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:00:05.544209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.550932+0000","last_change":"2023-03-03T06:26:57.543461+0000","last_active":"2023-03-03T06:26:57.550932+0000","last_peered":"2023-03-03T06:26:57.550932+0000","last_clean":"2023-03-03T06:26:57.550932+0000","last_became_active":"2023-03-03T06:26:57.543203+0000","last_became_peered":"2023-03-03T06:26:57.543203+0000","last_unstale":"2023-03-03T06:26:57.550932+0000","last_undegraded":"2023-03-03T06:26:57.550932+0000","last_fullsized":"2023-03-03T06:26:57.550932+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:33:49.811191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526662+0000","last_change":"2023-03-03T06:26:57.532765+0000","last_active":"2023-03-03T06:26:58.526662+0000","last_peered":"2023-03-03T06:26:58.526662+0000","last_clean":"2023-03-03T06:26:58.526662+0000","last_became_active":"2023-03-03T06:26:57.532476+0000","last_became_peered":"2023-03-03T06:26:57.532476+0000","last_unstale":"2023-03-03T06:26:58.526662+0000","last_undegraded":"2023-03-03T06:26:58.526662+0000","last_fullsized":"2023-03-03T06:26:58.526662+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:52:38.600353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.555974+0000","last_change":"2023-03-03T06:26:57.554709+0000","last_active":"2023-03-03T06:26:57.555974+0000","last_peered":"2023-03-03T06:26:57.555974+0000","last_clean":"2023-03-03T06:26:57.555974+0000","last_became_active":"2023-03-03T06:26:57.554350+0000","last_became_peered":"2023-03-03T06:26:57.554350+0000","last_unstale":"2023-03-03T06:26:57.555974+0000","last_undegraded":"2023-03-03T06:26:57.555974+0000","last_fullsized":"2023-03-03T06:26:57.555974+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:07:59.587114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.554753+0000","last_change":"2023-03-03T06:26:57.551033+0000","last_active":"2023-03-03T06:26:57.554753+0000","last_peered":"2023-03-03T06:26:57.554753+0000","last_clean":"2023-03-03T06:26:57.554753+0000","last_became_active":"2023-03-03T06:26:57.550377+0000","last_became_peered":"2023-03-03T06:26:57.550377+0000","last_unstale":"2023-03-03T06:26:57.554753+0000","last_undegraded":"2023-03-03T06:26:57.554753+0000","last_fullsized":"2023-03-03T06:26:57.554753+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:44:40.208203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.551947+0000","last_change":"2023-03-03T06:26:57.544468+0000","last_active":"2023-03-03T06:26:57.551947+0000","last_peered":"2023-03-03T06:26:57.551947+0000","last_clean":"2023-03-03T06:26:57.551947+0000","last_became_active":"2023-03-03T06:26:57.544183+0000","last_became_peered":"2023-03-03T06:26:57.544183+0000","last_unstale":"2023-03-03T06:26:57.551947+0000","last_undegraded":"2023-03-03T06:26:57.551947+0000","last_fullsized":"2023-03-03T06:26:57.551947+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:08:37.698278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.534698+0000","last_change":"2023-03-03T06:26:57.532648+0000","last_active":"2023-03-03T06:26:57.534698+0000","last_peered":"2023-03-03T06:26:57.534698+0000","last_clean":"2023-03-03T06:26:57.534698+0000","last_became_active":"2023-03-03T06:26:57.531621+0000","last_became_peered":"2023-03-03T06:26:57.531621+0000","last_unstale":"2023-03-03T06:26:57.534698+0000","last_undegraded":"2023-03-03T06:26:57.534698+0000","last_fullsized":"2023-03-03T06:26:57.534698+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:33:39.564321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.551432+0000","last_change":"2023-03-03T06:26:57.544796+0000","last_active":"2023-03-03T06:26:57.551432+0000","last_peered":"2023-03-03T06:26:57.551432+0000","last_clean":"2023-03-03T06:26:57.551432+0000","last_became_active":"2023-03-03T06:26:57.544261+0000","last_became_peered":"2023-03-03T06:26:57.544261+0000","last_unstale":"2023-03-03T06:26:57.551432+0000","last_undegraded":"2023-03-03T06:26:57.551432+0000","last_fullsized":"2023-03-03T06:26:57.551432+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:52:04.730707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.546697+0000","last_change":"2023-03-03T06:26:57.537160+0000","last_active":"2023-03-03T06:26:57.546697+0000","last_peered":"2023-03-03T06:26:57.546697+0000","last_clean":"2023-03-03T06:26:57.546697+0000","last_became_active":"2023-03-03T06:26:57.536927+0000","last_became_peered":"2023-03-03T06:26:57.536927+0000","last_unstale":"2023-03-03T06:26:57.546697+0000","last_undegraded":"2023-03-03T06:26:57.546697+0000","last_fullsized":"2023-03-03T06:26:57.546697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:52:38.600353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527620+0000","last_change":"2023-03-03T06:26:57.527963+0000","last_active":"2023-03-03T06:26:58.527620+0000","last_peered":"2023-03-03T06:26:58.527620+0000","last_clean":"2023-03-03T06:26:58.527620+0000","last_became_active":"2023-03-03T06:26:57.527354+0000","last_became_peered":"2023-03-03T06:26:57.527354+0000","last_unstale":"2023-03-03T06:26:58.527620+0000","last_undegraded":"2023-03-03T06:26:58.527620+0000","last_fullsized":"2023-03-03T06:26:58.527620+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:51:08.874774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.524933+0000","last_change":"2023-03-03T06:26:57.530138+0000","last_active":"2023-03-03T06:26:58.524933+0000","last_peered":"2023-03-03T06:26:58.524933+0000","last_clean":"2023-03-03T06:26:58.524933+0000","last_became_active":"2023-03-03T06:26:57.529800+0000","last_became_peered":"2023-03-03T06:26:57.529800+0000","last_unstale":"2023-03-03T06:26:58.524933+0000","last_undegraded":"2023-03-03T06:26:58.524933+0000","last_fullsized":"2023-03-03T06:26:58.524933+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:43:09.322094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.533606+0000","last_change":"2023-03-03T06:26:57.530356+0000","last_active":"2023-03-03T06:26:57.533606+0000","last_peered":"2023-03-03T06:26:57.533606+0000","last_clean":"2023-03-03T06:26:57.533606+0000","last_became_active":"2023-03-03T06:26:57.529492+0000","last_became_peered":"2023-03-03T06:26:57.529492+0000","last_unstale":"2023-03-03T06:26:57.533606+0000","last_undegraded":"2023-03-03T06:26:57.533606+0000","last_fullsized":"2023-03-03T06:26:57.533606+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:22:07.864707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.533230+0000","last_change":"2023-03-03T06:26:57.523517+0000","last_active":"2023-03-03T06:26:57.533230+0000","last_peered":"2023-03-03T06:26:57.533230+0000","last_clean":"2023-03-03T06:26:57.533230+0000","last_became_active":"2023-03-03T06:26:57.523256+0000","last_became_peered":"2023-03-03T06:26:57.523256+0000","last_unstale":"2023-03-03T06:26:57.533230+0000","last_undegraded":"2023-03-03T06:26:57.533230+0000","last_fullsized":"2023-03-03T06:26:57.533230+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:28:14.978083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.549335+0000","last_change":"2023-03-03T06:26:57.541331+0000","last_active":"2023-03-03T06:26:57.549335+0000","last_peered":"2023-03-03T06:26:57.549335+0000","last_clean":"2023-03-03T06:26:57.549335+0000","last_became_active":"2023-03-03T06:26:57.541053+0000","last_became_peered":"2023-03-03T06:26:57.541053+0000","last_unstale":"2023-03-03T06:26:57.549335+0000","last_undegraded":"2023-03-03T06:26:57.549335+0000","last_fullsized":"2023-03-03T06:26:57.549335+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:35:16.888845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527661+0000","last_change":"2023-03-03T06:26:57.521235+0000","last_active":"2023-03-03T06:26:58.527661+0000","last_peered":"2023-03-03T06:26:58.527661+0000","last_clean":"2023-03-03T06:26:58.527661+0000","last_became_active":"2023-03-03T06:26:57.520875+0000","last_became_peered":"2023-03-03T06:26:57.520875+0000","last_unstale":"2023-03-03T06:26:58.527661+0000","last_undegraded":"2023-03-03T06:26:58.527661+0000","last_fullsized":"2023-03-03T06:26:58.527661+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:33:32.980274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527160+0000","last_change":"2023-03-03T06:26:57.539157+0000","last_active":"2023-03-03T06:26:58.527160+0000","last_peered":"2023-03-03T06:26:58.527160+0000","last_clean":"2023-03-03T06:26:58.527160+0000","last_became_active":"2023-03-03T06:26:57.538499+0000","last_became_peered":"2023-03-03T06:26:57.538499+0000","last_unstale":"2023-03-03T06:26:58.527160+0000","last_undegraded":"2023-03-03T06:26:58.527160+0000","last_fullsized":"2023-03-03T06:26:58.527160+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:52:04.730707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.557646+0000","last_change":"2023-03-03T06:26:57.543141+0000","last_active":"2023-03-03T06:26:57.557646+0000","last_peered":"2023-03-03T06:26:57.557646+0000","last_clean":"2023-03-03T06:26:57.557646+0000","last_became_active":"2023-03-03T06:26:57.542748+0000","last_became_peered":"2023-03-03T06:26:57.542748+0000","last_unstale":"2023-03-03T06:26:57.557646+0000","last_undegraded":"2023-03-03T06:26:57.557646+0000","last_fullsized":"2023-03-03T06:26:57.557646+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:21:22.126399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.525087+0000","last_change":"2023-03-03T06:26:57.538527+0000","last_active":"2023-03-03T06:26:58.525087+0000","last_peered":"2023-03-03T06:26:58.525087+0000","last_clean":"2023-03-03T06:26:58.525087+0000","last_became_active":"2023-03-03T06:26:57.538169+0000","last_became_peered":"2023-03-03T06:26:57.538169+0000","last_unstale":"2023-03-03T06:26:58.525087+0000","last_undegraded":"2023-03-03T06:26:58.525087+0000","last_fullsized":"2023-03-03T06:26:58.525087+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:53:29.523443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.547137+0000","last_change":"2023-03-03T06:26:57.543402+0000","last_active":"2023-03-03T06:26:57.547137+0000","last_peered":"2023-03-03T06:26:57.547137+0000","last_clean":"2023-03-03T06:26:57.547137+0000","last_became_active":"2023-03-03T06:26:57.543152+0000","last_became_peered":"2023-03-03T06:26:57.543152+0000","last_unstale":"2023-03-03T06:26:57.547137+0000","last_undegraded":"2023-03-03T06:26:57.547137+0000","last_fullsized":"2023-03-03T06:26:57.547137+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:24:15.711569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.551028+0000","last_change":"2023-03-03T06:26:57.543683+0000","last_active":"2023-03-03T06:26:57.551028+0000","last_peered":"2023-03-03T06:26:57.551028+0000","last_clean":"2023-03-03T06:26:57.551028+0000","last_became_active":"2023-03-03T06:26:57.543378+0000","last_became_peered":"2023-03-03T06:26:57.543378+0000","last_unstale":"2023-03-03T06:26:57.551028+0000","last_undegraded":"2023-03-03T06:26:57.551028+0000","last_fullsized":"2023-03-03T06:26:57.551028+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:00:05.544209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.559502+0000","last_change":"2023-03-03T06:26:57.552792+0000","last_active":"2023-03-03T06:26:57.559502+0000","last_peered":"2023-03-03T06:26:57.559502+0000","last_clean":"2023-03-03T06:26:57.559502+0000","last_became_active":"2023-03-03T06:26:57.552414+0000","last_became_peered":"2023-03-03T06:26:57.552414+0000","last_unstale":"2023-03-03T06:26:57.559502+0000","last_undegraded":"2023-03-03T06:26:57.559502+0000","last_fullsized":"2023-03-03T06:26:57.559502+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:40:53.187472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.559464+0000","last_change":"2023-03-03T06:26:57.551622+0000","last_active":"2023-03-03T06:26:57.559464+0000","last_peered":"2023-03-03T06:26:57.559464+0000","last_clean":"2023-03-03T06:26:57.559464+0000","last_became_active":"2023-03-03T06:26:57.551346+0000","last_became_peered":"2023-03-03T06:26:57.551346+0000","last_unstale":"2023-03-03T06:26:57.559464+0000","last_undegraded":"2023-03-03T06:26:57.559464+0000","last_fullsized":"2023-03-03T06:26:57.559464+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:22:26.543385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.557764+0000","last_change":"2023-03-03T06:26:57.545789+0000","last_active":"2023-03-03T06:26:57.557764+0000","last_peered":"2023-03-03T06:26:57.557764+0000","last_clean":"2023-03-03T06:26:57.557764+0000","last_became_active":"2023-03-03T06:26:57.545236+0000","last_became_peered":"2023-03-03T06:26:57.545236+0000","last_unstale":"2023-03-03T06:26:57.557764+0000","last_undegraded":"2023-03-03T06:26:57.557764+0000","last_fullsized":"2023-03-03T06:26:57.557764+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:01:51.362166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.525116+0000","last_change":"2023-03-03T06:26:57.536273+0000","last_active":"2023-03-03T06:26:58.525116+0000","last_peered":"2023-03-03T06:26:58.525116+0000","last_clean":"2023-03-03T06:26:58.525116+0000","last_became_active":"2023-03-03T06:26:57.535981+0000","last_became_peered":"2023-03-03T06:26:57.535981+0000","last_unstale":"2023-03-03T06:26:58.525116+0000","last_undegraded":"2023-03-03T06:26:58.525116+0000","last_fullsized":"2023-03-03T06:26:58.525116+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:01:51.362166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526966+0000","last_change":"2023-03-03T06:26:57.525440+0000","last_active":"2023-03-03T06:26:58.526966+0000","last_peered":"2023-03-03T06:26:58.526966+0000","last_clean":"2023-03-03T06:26:58.526966+0000","last_became_active":"2023-03-03T06:26:57.525141+0000","last_became_peered":"2023-03-03T06:26:57.525141+0000","last_unstale":"2023-03-03T06:26:58.526966+0000","last_undegraded":"2023-03-03T06:26:58.526966+0000","last_fullsized":"2023-03-03T06:26:58.526966+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:22:07.864707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527302+0000","last_change":"2023-03-03T06:26:57.528132+0000","last_active":"2023-03-03T06:26:58.527302+0000","last_peered":"2023-03-03T06:26:58.527302+0000","last_clean":"2023-03-03T06:26:58.527302+0000","last_became_active":"2023-03-03T06:26:57.527678+0000","last_became_peered":"2023-03-03T06:26:57.527678+0000","last_unstale":"2023-03-03T06:26:58.527302+0000","last_undegraded":"2023-03-03T06:26:58.527302+0000","last_fullsized":"2023-03-03T06:26:58.527302+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:25:57.269311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.549166+0000","last_change":"2023-03-03T06:26:57.539824+0000","last_active":"2023-03-03T06:26:57.549166+0000","last_peered":"2023-03-03T06:26:57.549166+0000","last_clean":"2023-03-03T06:26:57.549166+0000","last_became_active":"2023-03-03T06:26:57.538206+0000","last_became_peered":"2023-03-03T06:26:57.538206+0000","last_unstale":"2023-03-03T06:26:57.549166+0000","last_undegraded":"2023-03-03T06:26:57.549166+0000","last_fullsized":"2023-03-03T06:26:57.549166+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:43:09.322094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526534+0000","last_change":"2023-03-03T06:26:57.552343+0000","last_active":"2023-03-03T06:26:58.526534+0000","last_peered":"2023-03-03T06:26:58.526534+0000","last_clean":"2023-03-03T06:26:58.526534+0000","last_became_active":"2023-03-03T06:26:57.552066+0000","last_became_peered":"2023-03-03T06:26:57.552066+0000","last_unstale":"2023-03-03T06:26:58.526534+0000","last_undegraded":"2023-03-03T06:26:58.526534+0000","last_fullsized":"2023-03-03T06:26:58.526534+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:33:39.564321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.533214+0000","last_change":"2023-03-03T06:26:57.523031+0000","last_active":"2023-03-03T06:26:57.533214+0000","last_peered":"2023-03-03T06:26:57.533214+0000","last_clean":"2023-03-03T06:26:57.533214+0000","last_became_active":"2023-03-03T06:26:57.522770+0000","last_became_peered":"2023-03-03T06:26:57.522770+0000","last_unstale":"2023-03-03T06:26:57.533214+0000","last_undegraded":"2023-03-03T06:26:57.533214+0000","last_fullsized":"2023-03-03T06:26:57.533214+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:12:36.268936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.524715+0000","last_change":"2023-03-03T06:26:57.537584+0000","last_active":"2023-03-03T06:26:58.524715+0000","last_peered":"2023-03-03T06:26:58.524715+0000","last_clean":"2023-03-03T06:26:58.524715+0000","last_became_active":"2023-03-03T06:26:57.532219+0000","last_became_peered":"2023-03-03T06:26:57.532219+0000","last_unstale":"2023-03-03T06:26:58.524715+0000","last_undegraded":"2023-03-03T06:26:58.524715+0000","last_fullsized":"2023-03-03T06:26:58.524715+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:47:58.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.535226+0000","last_change":"2023-03-03T06:26:57.528171+0000","last_active":"2023-03-03T06:26:57.535226+0000","last_peered":"2023-03-03T06:26:57.535226+0000","last_clean":"2023-03-03T06:26:57.535226+0000","last_became_active":"2023-03-03T06:26:57.527905+0000","last_became_peered":"2023-03-03T06:26:57.527905+0000","last_unstale":"2023-03-03T06:26:57.535226+0000","last_undegraded":"2023-03-03T06:26:57.535226+0000","last_fullsized":"2023-03-03T06:26:57.535226+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:18:43.324949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.559402+0000","last_change":"2023-03-03T06:26:57.009356+0000","last_active":"2023-03-03T06:26:57.559402+0000","last_peered":"2023-03-03T06:26:57.559402+0000","last_clean":"2023-03-03T06:26:57.559402+0000","last_became_active":"2023-03-03T06:26:57.009060+0000","last_became_peered":"2023-03-03T06:26:57.009060+0000","last_unstale":"2023-03-03T06:26:57.559402+0000","last_undegraded":"2023-03-03T06:26:57.559402+0000","last_fullsized":"2023-03-03T06:26:57.559402+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:26:53.257291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.557743+0000","last_change":"2023-03-03T06:26:57.545558+0000","last_active":"2023-03-03T06:26:57.557743+0000","last_peered":"2023-03-03T06:26:57.557743+0000","last_clean":"2023-03-03T06:26:57.557743+0000","last_became_active":"2023-03-03T06:26:57.544882+0000","last_became_peered":"2023-03-03T06:26:57.544882+0000","last_unstale":"2023-03-03T06:26:57.557743+0000","last_undegraded":"2023-03-03T06:26:57.557743+0000","last_fullsized":"2023-03-03T06:26:57.557743+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:47:58.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527312+0000","last_change":"2023-03-03T06:26:57.010540+0000","last_active":"2023-03-03T06:26:58.527312+0000","last_peered":"2023-03-03T06:26:58.527312+0000","last_clean":"2023-03-03T06:26:58.527312+0000","last_became_active":"2023-03-03T06:26:57.010119+0000","last_became_peered":"2023-03-03T06:26:57.010119+0000","last_unstale":"2023-03-03T06:26:58.527312+0000","last_undegraded":"2023-03-03T06:26:58.527312+0000","last_fullsized":"2023-03-03T06:26:58.527312+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:29:45.742607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527459+0000","last_change":"2023-03-03T06:26:57.530333+0000","last_active":"2023-03-03T06:26:58.527459+0000","last_peered":"2023-03-03T06:26:58.527459+0000","last_clean":"2023-03-03T06:26:58.527459+0000","last_became_active":"2023-03-03T06:26:57.529967+0000","last_became_peered":"2023-03-03T06:26:57.529967+0000","last_unstale":"2023-03-03T06:26:58.527459+0000","last_undegraded":"2023-03-03T06:26:58.527459+0000","last_fullsized":"2023-03-03T06:26:58.527459+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:44:40.208203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.541580+0000","last_change":"2023-03-03T06:26:57.000595+0000","last_active":"2023-03-03T06:26:57.541580+0000","last_peered":"2023-03-03T06:26:57.541580+0000","last_clean":"2023-03-03T06:26:57.541580+0000","last_became_active":"2023-03-03T06:26:56.999153+0000","last_became_peered":"2023-03-03T06:26:56.999153+0000","last_unstale":"2023-03-03T06:26:57.541580+0000","last_undegraded":"2023-03-03T06:26:57.541580+0000","last_fullsized":"2023-03-03T06:26:57.541580+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:47:57.848563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.547908+0000","last_change":"2023-03-03T06:26:57.538595+0000","last_active":"2023-03-03T06:26:57.547908+0000","last_peered":"2023-03-03T06:26:57.547908+0000","last_clean":"2023-03-03T06:26:57.547908+0000","last_became_active":"2023-03-03T06:26:57.538271+0000","last_became_peered":"2023-03-03T06:26:57.538271+0000","last_unstale":"2023-03-03T06:26:57.547908+0000","last_undegraded":"2023-03-03T06:26:57.547908+0000","last_fullsized":"2023-03-03T06:26:57.547908+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:01:02.587632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.554107+0000","last_change":"2023-03-03T06:26:56.997835+0000","last_active":"2023-03-03T06:26:57.554107+0000","last_peered":"2023-03-03T06:26:57.554107+0000","last_clean":"2023-03-03T06:26:57.554107+0000","last_became_active":"2023-03-03T06:26:56.997498+0000","last_became_peered":"2023-03-03T06:26:56.997498+0000","last_unstale":"2023-03-03T06:26:57.554107+0000","last_undegraded":"2023-03-03T06:26:57.554107+0000","last_fullsized":"2023-03-03T06:26:57.554107+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:40:52.170576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526400+0000","last_change":"2023-03-03T06:26:57.541244+0000","last_active":"2023-03-03T06:26:58.526400+0000","last_peered":"2023-03-03T06:26:58.526400+0000","last_clean":"2023-03-03T06:26:58.526400+0000","last_became_active":"2023-03-03T06:26:57.540994+0000","last_became_peered":"2023-03-03T06:26:57.540994+0000","last_unstale":"2023-03-03T06:26:58.526400+0000","last_undegraded":"2023-03-03T06:26:58.526400+0000","last_fullsized":"2023-03-03T06:26:58.526400+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:54:41.470293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.553556+0000","last_change":"2023-03-03T06:26:56.997151+0000","last_active":"2023-03-03T06:26:57.553556+0000","last_peered":"2023-03-03T06:26:57.553556+0000","last_clean":"2023-03-03T06:26:57.553556+0000","last_became_active":"2023-03-03T06:26:56.996608+0000","last_became_peered":"2023-03-03T06:26:56.996608+0000","last_unstale":"2023-03-03T06:26:57.553556+0000","last_undegraded":"2023-03-03T06:26:57.553556+0000","last_fullsized":"2023-03-03T06:26:57.553556+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T06:50:18.385759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.531988+0000","last_change":"2023-03-03T06:26:57.528809+0000","last_active":"2023-03-03T06:26:57.531988+0000","last_peered":"2023-03-03T06:26:57.531988+0000","last_clean":"2023-03-03T06:26:57.531988+0000","last_became_active":"2023-03-03T06:26:57.528423+0000","last_became_peered":"2023-03-03T06:26:57.528423+0000","last_unstale":"2023-03-03T06:26:57.531988+0000","last_undegraded":"2023-03-03T06:26:57.531988+0000","last_fullsized":"2023-03-03T06:26:57.531988+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:11:30.592668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526522+0000","last_change":"2023-03-03T06:26:56.999645+0000","last_active":"2023-03-03T06:26:58.526522+0000","last_peered":"2023-03-03T06:26:58.526522+0000","last_clean":"2023-03-03T06:26:58.526522+0000","last_became_active":"2023-03-03T06:26:56.999366+0000","last_became_peered":"2023-03-03T06:26:56.999366+0000","last_unstale":"2023-03-03T06:26:58.526522+0000","last_undegraded":"2023-03-03T06:26:58.526522+0000","last_fullsized":"2023-03-03T06:26:58.526522+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:09:06.442642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.551929+0000","last_change":"2023-03-03T06:26:57.544366+0000","last_active":"2023-03-03T06:26:57.551929+0000","last_peered":"2023-03-03T06:26:57.551929+0000","last_clean":"2023-03-03T06:26:57.551929+0000","last_became_active":"2023-03-03T06:26:57.543811+0000","last_became_peered":"2023-03-03T06:26:57.543811+0000","last_unstale":"2023-03-03T06:26:57.551929+0000","last_undegraded":"2023-03-03T06:26:57.551929+0000","last_fullsized":"2023-03-03T06:26:57.551929+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:54:41.470293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526683+0000","last_change":"2023-03-03T06:26:56.997092+0000","last_active":"2023-03-03T06:26:58.526683+0000","last_peered":"2023-03-03T06:26:58.526683+0000","last_clean":"2023-03-03T06:26:58.526683+0000","last_became_active":"2023-03-03T06:26:56.996785+0000","last_became_peered":"2023-03-03T06:26:56.996785+0000","last_unstale":"2023-03-03T06:26:58.526683+0000","last_undegraded":"2023-03-03T06:26:58.526683+0000","last_fullsized":"2023-03-03T06:26:58.526683+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:21:21.109503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.559483+0000","last_change":"2023-03-03T06:26:57.552371+0000","last_active":"2023-03-03T06:26:57.559483+0000","last_peered":"2023-03-03T06:26:57.559483+0000","last_clean":"2023-03-03T06:26:57.559483+0000","last_became_active":"2023-03-03T06:26:57.552059+0000","last_became_peered":"2023-03-03T06:26:57.552059+0000","last_unstale":"2023-03-03T06:26:57.559483+0000","last_undegraded":"2023-03-03T06:26:57.559483+0000","last_fullsized":"2023-03-03T06:26:57.559483+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:18:56.146005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.518819+0000","last_change":"2023-03-03T06:26:57.013410+0000","last_active":"2023-03-03T06:26:57.518819+0000","last_peered":"2023-03-03T06:26:57.518819+0000","last_clean":"2023-03-03T06:26:57.518819+0000","last_became_active":"2023-03-03T06:26:57.013127+0000","last_became_peered":"2023-03-03T06:26:57.013127+0000","last_unstale":"2023-03-03T06:26:57.518819+0000","last_undegraded":"2023-03-03T06:26:57.518819+0000","last_fullsized":"2023-03-03T06:26:57.518819+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:42:50.798978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527180+0000","last_change":"2023-03-03T06:26:57.539341+0000","last_active":"2023-03-03T06:26:58.527180+0000","last_peered":"2023-03-03T06:26:58.527180+0000","last_clean":"2023-03-03T06:26:58.527180+0000","last_became_active":"2023-03-03T06:26:57.538826+0000","last_became_peered":"2023-03-03T06:26:57.538826+0000","last_unstale":"2023-03-03T06:26:58.527180+0000","last_undegraded":"2023-03-03T06:26:58.527180+0000","last_fullsized":"2023-03-03T06:26:58.527180+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:33:49.811191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527529+0000","last_change":"2023-03-03T06:26:57.002111+0000","last_active":"2023-03-03T06:26:58.527529+0000","last_peered":"2023-03-03T06:26:58.527529+0000","last_clean":"2023-03-03T06:26:58.527529+0000","last_became_active":"2023-03-03T06:26:57.001827+0000","last_became_peered":"2023-03-03T06:26:57.001827+0000","last_unstale":"2023-03-03T06:26:58.527529+0000","last_undegraded":"2023-03-03T06:26:58.527529+0000","last_fullsized":"2023-03-03T06:26:58.527529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:52:37.583457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.554764+0000","last_change":"2023-03-03T06:26:57.550916+0000","last_active":"2023-03-03T06:26:57.554764+0000","last_peered":"2023-03-03T06:26:57.554764+0000","last_clean":"2023-03-03T06:26:57.554764+0000","last_became_active":"2023-03-03T06:26:57.550239+0000","last_became_peered":"2023-03-03T06:26:57.550239+0000","last_unstale":"2023-03-03T06:26:57.554764+0000","last_undegraded":"2023-03-03T06:26:57.554764+0000","last_fullsized":"2023-03-03T06:26:57.554764+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:51:08.874774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.522572+0000","last_change":"2023-03-03T06:26:57.006868+0000","last_active":"2023-03-03T06:26:57.522572+0000","last_peered":"2023-03-03T06:26:57.522572+0000","last_clean":"2023-03-03T06:26:57.522572+0000","last_became_active":"2023-03-03T06:26:57.003408+0000","last_became_peered":"2023-03-03T06:26:57.003408+0000","last_unstale":"2023-03-03T06:26:57.522572+0000","last_undegraded":"2023-03-03T06:26:57.522572+0000","last_fullsized":"2023-03-03T06:26:57.522572+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:18:55.129109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.535402+0000","last_change":"2023-03-03T06:26:57.527926+0000","last_active":"2023-03-03T06:26:57.535402+0000","last_peered":"2023-03-03T06:26:57.535402+0000","last_clean":"2023-03-03T06:26:57.535402+0000","last_became_active":"2023-03-03T06:26:57.527475+0000","last_became_peered":"2023-03-03T06:26:57.527475+0000","last_unstale":"2023-03-03T06:26:57.535402+0000","last_undegraded":"2023-03-03T06:26:57.535402+0000","last_fullsized":"2023-03-03T06:26:57.535402+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:02:20.606207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.558323+0000","last_change":"2023-03-03T06:26:57.009453+0000","last_active":"2023-03-03T06:26:57.558323+0000","last_peered":"2023-03-03T06:26:57.558323+0000","last_clean":"2023-03-03T06:26:57.558323+0000","last_became_active":"2023-03-03T06:26:57.009194+0000","last_became_peered":"2023-03-03T06:26:57.009194+0000","last_unstale":"2023-03-03T06:26:57.558323+0000","last_undegraded":"2023-03-03T06:26:57.558323+0000","last_fullsized":"2023-03-03T06:26:57.558323+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:18:42.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.552789+0000","last_change":"2023-03-03T06:26:57.548226+0000","last_active":"2023-03-03T06:26:57.552789+0000","last_peered":"2023-03-03T06:26:57.552789+0000","last_clean":"2023-03-03T06:26:57.552789+0000","last_became_active":"2023-03-03T06:26:57.547920+0000","last_became_peered":"2023-03-03T06:26:57.547920+0000","last_unstale":"2023-03-03T06:26:57.552789+0000","last_undegraded":"2023-03-03T06:26:57.552789+0000","last_fullsized":"2023-03-03T06:26:57.552789+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:26:54.274187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.525104+0000","last_change":"2023-03-03T06:26:57.006387+0000","last_active":"2023-03-03T06:26:58.525104+0000","last_peered":"2023-03-03T06:26:58.525104+0000","last_clean":"2023-03-03T06:26:58.525104+0000","last_became_active":"2023-03-03T06:26:57.006088+0000","last_became_peered":"2023-03-03T06:26:57.006088+0000","last_unstale":"2023-03-03T06:26:58.525104+0000","last_undegraded":"2023-03-03T06:26:58.525104+0000","last_fullsized":"2023-03-03T06:26:58.525104+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:27:52.947467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.557764+0000","last_change":"2023-03-03T06:26:57.545237+0000","last_active":"2023-03-03T06:26:57.557764+0000","last_peered":"2023-03-03T06:26:57.557764+0000","last_clean":"2023-03-03T06:26:57.557764+0000","last_became_active":"2023-03-03T06:26:57.544543+0000","last_became_peered":"2023-03-03T06:26:57.544543+0000","last_unstale":"2023-03-03T06:26:57.557764+0000","last_undegraded":"2023-03-03T06:26:57.557764+0000","last_fullsized":"2023-03-03T06:26:57.557764+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:42:51.815874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.553243+0000","last_change":"2023-03-03T06:26:57.005116+0000","last_active":"2023-03-03T06:26:57.553243+0000","last_peered":"2023-03-03T06:26:57.553243+0000","last_clean":"2023-03-03T06:26:57.553243+0000","last_became_active":"2023-03-03T06:26:57.004870+0000","last_became_peered":"2023-03-03T06:26:57.004870+0000","last_unstale":"2023-03-03T06:26:57.553243+0000","last_undegraded":"2023-03-03T06:26:57.553243+0000","last_fullsized":"2023-03-03T06:26:57.553243+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:34:53.416403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.555350+0000","last_change":"2023-03-03T06:26:57.551643+0000","last_active":"2023-03-03T06:26:57.555350+0000","last_peered":"2023-03-03T06:26:57.555350+0000","last_clean":"2023-03-03T06:26:57.555350+0000","last_became_active":"2023-03-03T06:26:57.551348+0000","last_became_peered":"2023-03-03T06:26:57.551348+0000","last_unstale":"2023-03-03T06:26:57.555350+0000","last_undegraded":"2023-03-03T06:26:57.555350+0000","last_fullsized":"2023-03-03T06:26:57.555350+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T11:33:32.980274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.537206+0000","last_change":"2023-03-03T06:26:57.002269+0000","last_active":"2023-03-03T06:26:57.537206+0000","last_peered":"2023-03-03T06:26:57.537206+0000","last_clean":"2023-03-03T06:26:57.537206+0000","last_became_active":"2023-03-03T06:26:57.001962+0000","last_became_peered":"2023-03-03T06:26:57.001962+0000","last_unstale":"2023-03-03T06:26:57.537206+0000","last_undegraded":"2023-03-03T06:26:57.537206+0000","last_fullsized":"2023-03-03T06:26:57.537206+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:35:15.871949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.533187+0000","last_change":"2023-03-03T06:26:57.527074+0000","last_active":"2023-03-03T06:26:57.533187+0000","last_peered":"2023-03-03T06:26:57.533187+0000","last_clean":"2023-03-03T06:26:57.533187+0000","last_became_active":"2023-03-03T06:26:57.526791+0000","last_became_peered":"2023-03-03T06:26:57.526791+0000","last_unstale":"2023-03-03T06:26:57.533187+0000","last_undegraded":"2023-03-03T06:26:57.533187+0000","last_fullsized":"2023-03-03T06:26:57.533187+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:16:19.516452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.513832+0000","last_change":"2023-03-03T06:26:57.012556+0000","last_active":"2023-03-03T06:26:57.513832+0000","last_peered":"2023-03-03T06:26:57.513832+0000","last_clean":"2023-03-03T06:26:57.513832+0000","last_became_active":"2023-03-03T06:26:57.012011+0000","last_became_peered":"2023-03-03T06:26:57.012011+0000","last_unstale":"2023-03-03T06:26:57.513832+0000","last_undegraded":"2023-03-03T06:26:57.513832+0000","last_fullsized":"2023-03-03T06:26:57.513832+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:58:42.202828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.554775+0000","last_change":"2023-03-03T06:26:57.551237+0000","last_active":"2023-03-03T06:26:57.554775+0000","last_peered":"2023-03-03T06:26:57.554775+0000","last_clean":"2023-03-03T06:26:57.554775+0000","last_became_active":"2023-03-03T06:26:57.550693+0000","last_became_peered":"2023-03-03T06:26:57.550693+0000","last_unstale":"2023-03-03T06:26:57.554775+0000","last_undegraded":"2023-03-03T06:26:57.554775+0000","last_fullsized":"2023-03-03T06:26:57.554775+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:31:36.667958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.555442+0000","last_change":"2023-03-03T06:26:57.011832+0000","last_active":"2023-03-03T06:26:57.555442+0000","last_peered":"2023-03-03T06:26:57.555442+0000","last_clean":"2023-03-03T06:26:57.555442+0000","last_became_active":"2023-03-03T06:26:57.011552+0000","last_became_peered":"2023-03-03T06:26:57.011552+0000","last_unstale":"2023-03-03T06:26:57.555442+0000","last_undegraded":"2023-03-03T06:26:57.555442+0000","last_fullsized":"2023-03-03T06:26:57.555442+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:21:21.109503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.525204+0000","last_change":"2023-03-03T06:26:57.522979+0000","last_active":"2023-03-03T06:26:58.525204+0000","last_peered":"2023-03-03T06:26:58.525204+0000","last_clean":"2023-03-03T06:26:58.525204+0000","last_became_active":"2023-03-03T06:26:57.522737+0000","last_became_peered":"2023-03-03T06:26:57.522737+0000","last_unstale":"2023-03-03T06:26:58.525204+0000","last_undegraded":"2023-03-03T06:26:58.525204+0000","last_fullsized":"2023-03-03T06:26:58.525204+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:22:26.543385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.525285+0000","last_change":"2023-03-03T06:26:57.004509+0000","last_active":"2023-03-03T06:26:58.525285+0000","last_peered":"2023-03-03T06:26:58.525285+0000","last_clean":"2023-03-03T06:26:58.525285+0000","last_became_active":"2023-03-03T06:26:57.004197+0000","last_became_peered":"2023-03-03T06:26:57.004197+0000","last_unstale":"2023-03-03T06:26:58.525285+0000","last_undegraded":"2023-03-03T06:26:58.525285+0000","last_fullsized":"2023-03-03T06:26:58.525285+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:10:41.501137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.532548+0000","last_change":"2023-03-03T06:26:57.528425+0000","last_active":"2023-03-03T06:26:57.532548+0000","last_peered":"2023-03-03T06:26:57.532548+0000","last_clean":"2023-03-03T06:26:57.532548+0000","last_became_active":"2023-03-03T06:26:57.528057+0000","last_became_peered":"2023-03-03T06:26:57.528057+0000","last_unstale":"2023-03-03T06:26:57.532548+0000","last_undegraded":"2023-03-03T06:26:57.532548+0000","last_fullsized":"2023-03-03T06:26:57.532548+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:58:43.219725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527431+0000","last_change":"2023-03-03T06:26:56.995475+0000","last_active":"2023-03-03T06:26:58.527431+0000","last_peered":"2023-03-03T06:26:58.527431+0000","last_clean":"2023-03-03T06:26:58.527431+0000","last_became_active":"2023-03-03T06:26:56.995200+0000","last_became_peered":"2023-03-03T06:26:56.995200+0000","last_unstale":"2023-03-03T06:26:58.527431+0000","last_undegraded":"2023-03-03T06:26:58.527431+0000","last_fullsized":"2023-03-03T06:26:58.527431+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:36:16.195494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.549193+0000","last_change":"2023-03-03T06:26:57.541024+0000","last_active":"2023-03-03T06:26:57.549193+0000","last_peered":"2023-03-03T06:26:57.549193+0000","last_clean":"2023-03-03T06:26:57.549193+0000","last_became_active":"2023-03-03T06:26:57.540744+0000","last_became_peered":"2023-03-03T06:26:57.540744+0000","last_unstale":"2023-03-03T06:26:57.549193+0000","last_undegraded":"2023-03-03T06:26:57.549193+0000","last_fullsized":"2023-03-03T06:26:57.549193+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:18:43.324949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527521+0000","last_change":"2023-03-03T06:26:56.996779+0000","last_active":"2023-03-03T06:26:58.527521+0000","last_peered":"2023-03-03T06:26:58.527521+0000","last_clean":"2023-03-03T06:26:58.527521+0000","last_became_active":"2023-03-03T06:26:56.996489+0000","last_became_peered":"2023-03-03T06:26:56.996489+0000","last_unstale":"2023-03-03T06:26:58.527521+0000","last_undegraded":"2023-03-03T06:26:58.527521+0000","last_fullsized":"2023-03-03T06:26:58.527521+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:42:50.798978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.559429+0000","last_change":"2023-03-03T06:26:57.550703+0000","last_active":"2023-03-03T06:26:57.559429+0000","last_peered":"2023-03-03T06:26:57.559429+0000","last_clean":"2023-03-03T06:26:57.559429+0000","last_became_active":"2023-03-03T06:26:57.548830+0000","last_became_peered":"2023-03-03T06:26:57.548830+0000","last_unstale":"2023-03-03T06:26:57.559429+0000","last_undegraded":"2023-03-03T06:26:57.559429+0000","last_fullsized":"2023-03-03T06:26:57.559429+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:53:29.523443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.516945+0000","last_change":"2023-03-03T06:26:57.009951+0000","last_active":"2023-03-03T06:26:57.516945+0000","last_peered":"2023-03-03T06:26:57.516945+0000","last_clean":"2023-03-03T06:26:57.516945+0000","last_became_active":"2023-03-03T06:26:57.009684+0000","last_became_peered":"2023-03-03T06:26:57.009684+0000","last_unstale":"2023-03-03T06:26:57.516945+0000","last_undegraded":"2023-03-03T06:26:57.516945+0000","last_fullsized":"2023-03-03T06:26:57.516945+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:44:39.191307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.533175+0000","last_change":"2023-03-03T06:26:57.527276+0000","last_active":"2023-03-03T06:26:57.533175+0000","last_peered":"2023-03-03T06:26:57.533175+0000","last_clean":"2023-03-03T06:26:57.533175+0000","last_became_active":"2023-03-03T06:26:57.526961+0000","last_became_peered":"2023-03-03T06:26:57.526961+0000","last_unstale":"2023-03-03T06:26:57.533175+0000","last_undegraded":"2023-03-03T06:26:57.533175+0000","last_fullsized":"2023-03-03T06:26:57.533175+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:00:24.063719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.553718+0000","last_change":"2023-03-03T06:26:56.997184+0000","last_active":"2023-03-03T06:26:57.553718+0000","last_peered":"2023-03-03T06:26:57.553718+0000","last_clean":"2023-03-03T06:26:57.553718+0000","last_became_active":"2023-03-03T06:26:56.996859+0000","last_became_peered":"2023-03-03T06:26:56.996859+0000","last_unstale":"2023-03-03T06:26:57.553718+0000","last_undegraded":"2023-03-03T06:26:57.553718+0000","last_fullsized":"2023-03-03T06:26:57.553718+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T12:31:35.651062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.529820+0000","last_change":"2023-03-03T06:26:57.521350+0000","last_active":"2023-03-03T06:26:57.529820+0000","last_peered":"2023-03-03T06:26:57.529820+0000","last_clean":"2023-03-03T06:26:57.529820+0000","last_became_active":"2023-03-03T06:26:57.521082+0000","last_became_peered":"2023-03-03T06:26:57.521082+0000","last_unstale":"2023-03-03T06:26:57.529820+0000","last_undegraded":"2023-03-03T06:26:57.529820+0000","last_fullsized":"2023-03-03T06:26:57.529820+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:42:51.815874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526767+0000","last_change":"2023-03-03T06:26:57.010508+0000","last_active":"2023-03-03T06:26:58.526767+0000","last_peered":"2023-03-03T06:26:58.526767+0000","last_clean":"2023-03-03T06:26:58.526767+0000","last_became_active":"2023-03-03T06:26:57.010216+0000","last_became_peered":"2023-03-03T06:26:57.010216+0000","last_unstale":"2023-03-03T06:26:58.526767+0000","last_undegraded":"2023-03-03T06:26:58.526767+0000","last_fullsized":"2023-03-03T06:26:58.526767+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:40:52.170576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.524794+0000","last_change":"2023-03-03T06:26:57.551051+0000","last_active":"2023-03-03T06:26:58.524794+0000","last_peered":"2023-03-03T06:26:58.524794+0000","last_clean":"2023-03-03T06:26:58.524794+0000","last_became_active":"2023-03-03T06:26:57.550590+0000","last_became_peered":"2023-03-03T06:26:57.550590+0000","last_unstale":"2023-03-03T06:26:58.524794+0000","last_undegraded":"2023-03-03T06:26:58.524794+0000","last_fullsized":"2023-03-03T06:26:58.524794+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:18:56.146005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.519265+0000","last_change":"2023-03-03T06:26:57.011481+0000","last_active":"2023-03-03T06:26:57.519265+0000","last_peered":"2023-03-03T06:26:57.519265+0000","last_clean":"2023-03-03T06:26:57.519265+0000","last_became_active":"2023-03-03T06:26:57.011012+0000","last_became_peered":"2023-03-03T06:26:57.011012+0000","last_unstale":"2023-03-03T06:26:57.519265+0000","last_undegraded":"2023-03-03T06:26:57.519265+0000","last_fullsized":"2023-03-03T06:26:57.519265+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:40:52.170576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.546546+0000","last_change":"2023-03-03T06:26:57.536646+0000","last_active":"2023-03-03T06:26:57.546546+0000","last_peered":"2023-03-03T06:26:57.546546+0000","last_clean":"2023-03-03T06:26:57.546546+0000","last_became_active":"2023-03-03T06:26:57.536386+0000","last_became_peered":"2023-03-03T06:26:57.536386+0000","last_unstale":"2023-03-03T06:26:57.546546+0000","last_undegraded":"2023-03-03T06:26:57.546546+0000","last_fullsized":"2023-03-03T06:26:57.546546+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:58:45.161662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.534425+0000","last_change":"2023-03-03T06:26:57.003280+0000","last_active":"2023-03-03T06:26:57.534425+0000","last_peered":"2023-03-03T06:26:57.534425+0000","last_clean":"2023-03-03T06:26:57.534425+0000","last_became_active":"2023-03-03T06:26:57.002201+0000","last_became_peered":"2023-03-03T06:26:57.002201+0000","last_unstale":"2023-03-03T06:26:57.534425+0000","last_undegraded":"2023-03-03T06:26:57.534425+0000","last_fullsized":"2023-03-03T06:26:57.534425+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:01:01.570736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.533154+0000","last_change":"2023-03-03T06:26:57.524538+0000","last_active":"2023-03-03T06:26:57.533154+0000","last_peered":"2023-03-03T06:26:57.533154+0000","last_clean":"2023-03-03T06:26:57.533154+0000","last_became_active":"2023-03-03T06:26:57.524260+0000","last_became_peered":"2023-03-03T06:26:57.524260+0000","last_unstale":"2023-03-03T06:26:57.533154+0000","last_undegraded":"2023-03-03T06:26:57.533154+0000","last_fullsized":"2023-03-03T06:26:57.533154+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:21:27.992654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.536393+0000","last_change":"2023-03-03T06:26:57.007428+0000","last_active":"2023-03-03T06:26:57.536393+0000","last_peered":"2023-03-03T06:26:57.536393+0000","last_clean":"2023-03-03T06:26:57.536393+0000","last_became_active":"2023-03-03T06:26:57.007065+0000","last_became_peered":"2023-03-03T06:26:57.007065+0000","last_unstale":"2023-03-03T06:26:57.536393+0000","last_undegraded":"2023-03-03T06:26:57.536393+0000","last_fullsized":"2023-03-03T06:26:57.536393+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T07:42:50.798978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.003331+0000","last_change":"2023-03-03T06:25:07.205700+0000","last_active":"2023-03-03T06:26:57.003331+0000","last_peered":"2023-03-03T06:26:57.003331+0000","last_clean":"2023-03-03T06:26:57.003331+0000","last_became_active":"2023-03-03T06:25:07.205269+0000","last_became_peered":"2023-03-03T06:25:07.205269+0000","last_unstale":"2023-03-03T06:26:57.003331+0000","last_undegraded":"2023-03-03T06:26:57.003331+0000","last_fullsized":"2023-03-03T06:26:57.003331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:17:53.442570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.537293+0000","last_change":"2023-03-03T06:26:57.531174+0000","last_active":"2023-03-03T06:26:57.537293+0000","last_peered":"2023-03-03T06:26:57.537293+0000","last_clean":"2023-03-03T06:26:57.537293+0000","last_became_active":"2023-03-03T06:26:57.530872+0000","last_became_peered":"2023-03-03T06:26:57.530872+0000","last_unstale":"2023-03-03T06:26:57.537293+0000","last_undegraded":"2023-03-03T06:26:57.537293+0000","last_fullsized":"2023-03-03T06:26:57.537293+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:42:05.211539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526625+0000","last_change":"2023-03-03T06:26:56.997545+0000","last_active":"2023-03-03T06:26:58.526625+0000","last_peered":"2023-03-03T06:26:58.526625+0000","last_clean":"2023-03-03T06:26:58.526625+0000","last_became_active":"2023-03-03T06:26:56.997258+0000","last_became_peered":"2023-03-03T06:26:56.997258+0000","last_unstale":"2023-03-03T06:26:58.526625+0000","last_undegraded":"2023-03-03T06:26:58.526625+0000","last_fullsized":"2023-03-03T06:26:58.526625+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:23:12.695056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-03T06:26:56.990738+0000","last_change":"2023-03-03T06:25:07.204903+0000","last_active":"2023-03-03T06:26:56.990738+0000","last_peered":"2023-03-03T06:26:56.990738+0000","last_clean":"2023-03-03T06:26:56.990738+0000","last_became_active":"2023-03-03T06:25:07.204601+0000","last_became_peered":"2023-03-03T06:25:07.204601+0000","last_unstale":"2023-03-03T06:26:56.990738+0000","last_undegraded":"2023-03-03T06:26:56.990738+0000","last_fullsized":"2023-03-03T06:26:56.990738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:18:36.947912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526861+0000","last_change":"2023-03-03T06:26:57.529626+0000","last_active":"2023-03-03T06:26:58.526861+0000","last_peered":"2023-03-03T06:26:58.526861+0000","last_clean":"2023-03-03T06:26:58.526861+0000","last_became_active":"2023-03-03T06:26:57.529015+0000","last_became_peered":"2023-03-03T06:26:57.529015+0000","last_unstale":"2023-03-03T06:26:58.526861+0000","last_undegraded":"2023-03-03T06:26:58.526861+0000","last_fullsized":"2023-03-03T06:26:58.526861+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:29:46.759504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.510527+0000","last_change":"2023-03-03T06:26:57.010209+0000","last_active":"2023-03-03T06:26:57.510527+0000","last_peered":"2023-03-03T06:26:57.510527+0000","last_clean":"2023-03-03T06:26:57.510527+0000","last_became_active":"2023-03-03T06:26:57.009881+0000","last_became_peered":"2023-03-03T06:26:57.009881+0000","last_unstale":"2023-03-03T06:26:57.510527+0000","last_undegraded":"2023-03-03T06:26:57.510527+0000","last_fullsized":"2023-03-03T06:26:57.510527+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T14:43:08.305198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-03T06:26:56.990475+0000","last_change":"2023-03-03T06:25:07.208473+0000","last_active":"2023-03-03T06:26:56.990475+0000","last_peered":"2023-03-03T06:26:56.990475+0000","last_clean":"2023-03-03T06:26:56.990475+0000","last_became_active":"2023-03-03T06:25:07.208041+0000","last_became_peered":"2023-03-03T06:25:07.208041+0000","last_unstale":"2023-03-03T06:26:56.990475+0000","last_undegraded":"2023-03-03T06:26:56.990475+0000","last_fullsized":"2023-03-03T06:26:56.990475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:22:19.852493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.526375+0000","last_change":"2023-03-03T06:26:57.530454+0000","last_active":"2023-03-03T06:26:58.526375+0000","last_peered":"2023-03-03T06:26:58.526375+0000","last_clean":"2023-03-03T06:26:58.526375+0000","last_became_active":"2023-03-03T06:26:57.530101+0000","last_became_peered":"2023-03-03T06:26:57.530101+0000","last_unstale":"2023-03-03T06:26:58.526375+0000","last_undegraded":"2023-03-03T06:26:58.526375+0000","last_fullsized":"2023-03-03T06:26:58.526375+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:38:20.227293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.518909+0000","last_change":"2023-03-03T06:26:57.011360+0000","last_active":"2023-03-03T06:26:57.518909+0000","last_peered":"2023-03-03T06:26:57.518909+0000","last_clean":"2023-03-03T06:26:57.518909+0000","last_became_active":"2023-03-03T06:26:57.010981+0000","last_became_peered":"2023-03-03T06:26:57.010981+0000","last_unstale":"2023-03-03T06:26:57.518909+0000","last_undegraded":"2023-03-03T06:26:57.518909+0000","last_fullsized":"2023-03-03T06:26:57.518909+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:22:25.526489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-03T06:26:56.991460+0000","last_change":"2023-03-03T06:25:07.201865+0000","last_active":"2023-03-03T06:26:56.991460+0000","last_peered":"2023-03-03T06:26:56.991460+0000","last_clean":"2023-03-03T06:26:56.991460+0000","last_became_active":"2023-03-03T06:25:07.201411+0000","last_became_peered":"2023-03-03T06:25:07.201411+0000","last_unstale":"2023-03-03T06:26:56.991460+0000","last_undegraded":"2023-03-03T06:26:56.991460+0000","last_fullsized":"2023-03-03T06:26:56.991460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T15:17:53.442570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.557449+0000","last_change":"2023-03-03T06:26:57.543371+0000","last_active":"2023-03-03T06:26:57.557449+0000","last_peered":"2023-03-03T06:26:57.557449+0000","last_clean":"2023-03-03T06:26:57.557449+0000","last_became_active":"2023-03-03T06:26:57.543067+0000","last_became_peered":"2023-03-03T06:26:57.543067+0000","last_unstale":"2023-03-03T06:26:57.557449+0000","last_undegraded":"2023-03-03T06:26:57.557449+0000","last_fullsized":"2023-03-03T06:26:57.557449+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:01:02.587632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.515339+0000","last_change":"2023-03-03T06:26:57.001058+0000","last_active":"2023-03-03T06:26:57.515339+0000","last_peered":"2023-03-03T06:26:57.515339+0000","last_clean":"2023-03-03T06:26:57.515339+0000","last_became_active":"2023-03-03T06:26:57.000798+0000","last_became_peered":"2023-03-03T06:26:57.000798+0000","last_unstale":"2023-03-03T06:26:57.515339+0000","last_undegraded":"2023-03-03T06:26:57.515339+0000","last_fullsized":"2023-03-03T06:26:57.515339+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:25:56.252415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.513829+0000","last_change":"2023-03-03T06:25:07.201941+0000","last_active":"2023-03-03T06:26:57.513829+0000","last_peered":"2023-03-03T06:26:57.513829+0000","last_clean":"2023-03-03T06:26:57.513829+0000","last_became_active":"2023-03-03T06:25:07.201437+0000","last_became_peered":"2023-03-03T06:25:07.201437+0000","last_unstale":"2023-03-03T06:26:57.513829+0000","last_undegraded":"2023-03-03T06:26:57.513829+0000","last_fullsized":"2023-03-03T06:26:57.513829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:48:52.145810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.543721+0000","last_change":"2023-03-03T06:26:57.534202+0000","last_active":"2023-03-03T06:26:57.543721+0000","last_peered":"2023-03-03T06:26:57.543721+0000","last_clean":"2023-03-03T06:26:57.543721+0000","last_became_active":"2023-03-03T06:26:57.533864+0000","last_became_peered":"2023-03-03T06:26:57.533864+0000","last_unstale":"2023-03-03T06:26:57.543721+0000","last_undegraded":"2023-03-03T06:26:57.543721+0000","last_fullsized":"2023-03-03T06:26:57.543721+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:38:20.227293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.531866+0000","last_change":"2023-03-03T06:26:57.012078+0000","last_active":"2023-03-03T06:26:57.531866+0000","last_peered":"2023-03-03T06:26:57.531866+0000","last_clean":"2023-03-03T06:26:57.531866+0000","last_became_active":"2023-03-03T06:26:57.011761+0000","last_became_peered":"2023-03-03T06:26:57.011761+0000","last_unstale":"2023-03-03T06:26:57.531866+0000","last_undegraded":"2023-03-03T06:26:57.531866+0000","last_fullsized":"2023-03-03T06:26:57.531866+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:21:21.109503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.514710+0000","last_change":"2023-03-03T06:25:07.437275+0000","last_active":"2023-03-03T06:26:57.514710+0000","last_peered":"2023-03-03T06:26:57.514710+0000","last_clean":"2023-03-03T06:26:57.514710+0000","last_became_active":"2023-03-03T06:25:07.436823+0000","last_became_peered":"2023-03-03T06:25:07.436823+0000","last_unstale":"2023-03-03T06:26:57.514710+0000","last_undegraded":"2023-03-03T06:26:57.514710+0000","last_fullsized":"2023-03-03T06:26:57.514710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T09:24:15.529041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.555931+0000","last_change":"2023-03-03T06:26:57.552842+0000","last_active":"2023-03-03T06:26:57.555931+0000","last_peered":"2023-03-03T06:26:57.555931+0000","last_clean":"2023-03-03T06:26:57.555931+0000","last_became_active":"2023-03-03T06:26:57.552107+0000","last_became_peered":"2023-03-03T06:26:57.552107+0000","last_unstale":"2023-03-03T06:26:57.555931+0000","last_undegraded":"2023-03-03T06:26:57.555931+0000","last_fullsized":"2023-03-03T06:26:57.555931+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T13:34:54.433299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.511130+0000","last_change":"2023-03-03T06:26:57.005618+0000","last_active":"2023-03-03T06:26:57.511130+0000","last_peered":"2023-03-03T06:26:57.511130+0000","last_clean":"2023-03-03T06:26:57.511130+0000","last_became_active":"2023-03-03T06:26:57.005211+0000","last_became_peered":"2023-03-03T06:26:57.005211+0000","last_unstale":"2023-03-03T06:26:57.511130+0000","last_undegraded":"2023-03-03T06:26:57.511130+0000","last_fullsized":"2023-03-03T06:26:57.511130+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T17:53:28.506546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-03T06:26:56.983616+0000","last_change":"2023-03-03T06:25:07.208585+0000","last_active":"2023-03-03T06:26:56.983616+0000","last_peered":"2023-03-03T06:26:56.983616+0000","last_clean":"2023-03-03T06:26:56.983616+0000","last_became_active":"2023-03-03T06:25:07.208257+0000","last_became_peered":"2023-03-03T06:25:07.208257+0000","last_unstale":"2023-03-03T06:26:56.983616+0000","last_undegraded":"2023-03-03T06:26:56.983616+0000","last_fullsized":"2023-03-03T06:26:56.983616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T08:11:52.893733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.555324+0000","last_change":"2023-03-03T06:26:57.551543+0000","last_active":"2023-03-03T06:26:57.555324+0000","last_peered":"2023-03-03T06:26:57.555324+0000","last_clean":"2023-03-03T06:26:57.555324+0000","last_became_active":"2023-03-03T06:26:57.551250+0000","last_became_peered":"2023-03-03T06:26:57.551250+0000","last_unstale":"2023-03-03T06:26:57.555324+0000","last_undegraded":"2023-03-03T06:26:57.555324+0000","last_fullsized":"2023-03-03T06:26:57.555324+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:56.959487+0000","last_clean_scrub_stamp":"2023-03-03T06:26:56.959487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T18:26:54.274187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-03T06:26:58.527623+0000","last_change":"2023-03-03T06:26:57.002442+0000","last_active":"2023-03-03T06:26:58.527623+0000","last_peered":"2023-03-03T06:26:58.527623+0000","last_clean":"2023-03-03T06:26:58.527623+0000","last_became_active":"2023-03-03T06:26:57.002158+0000","last_became_peered":"2023-03-03T06:26:57.002158+0000","last_unstale":"2023-03-03T06:26:58.527623+0000","last_undegraded":"2023-03-03T06:26:58.527623+0000","last_fullsized":"2023-03-03T06:26:58.527623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:26:55.942591+0000","last_clean_scrub_stamp":"2023-03-03T06:26:55.942591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:18:55.129109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-03T06:26:57.517767+0000","last_change":"2023-03-03T06:25:07.203115+0000","last_active":"2023-03-03T06:26:57.517767+0000","last_peered":"2023-03-03T06:26:57.517767+0000","last_clean":"2023-03-03T06:26:57.517767+0000","last_became_active":"2023-03-03T06:25:07.202705+0000","last_became_peered":"2023-03-03T06:25:07.202705+0000","last_unstale":"2023-03-03T06:26:57.517767+0000","last_undegraded":"2023-03-03T06:26:57.517767+0000","last_fullsized":"2023-03-03T06:26:57.517767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:06.166360+0000","last_clean_scrub_stamp":"2023-03-03T06:25:06.166360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T10:11:32.938698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":183,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-03-03T06:26:56.984455+0000","last_change":"2023-03-03T06:25:02.138688+0000","last_active":"2023-03-03T06:26:56.984455+0000","last_peered":"2023-03-03T06:26:56.984455+0000","last_clean":"2023-03-03T06:26:56.984455+0000","last_became_active":"2023-03-03T06:25:02.137945+0000","last_became_peered":"2023-03-03T06:25:02.137945+0000","last_unstale":"2023-03-03T06:26:56.984455+0000","last_undegraded":"2023-03-03T06:26:56.984455+0000","last_fullsized":"2023-03-03T06:26:56.984455+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-03T06:25:00.679233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-03T06:25:00.679233+0000","last_clean_scrub_stamp":"2023-03-03T06:25:00.679233+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-04T16:24:46.304379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13932,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94357908,"statfs":{"total":96636764160,"available":96622497792,"internally_reserved":0,"allocated":1085440,"data_stored":806932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.4980000000000002,"5min":2.4980000000000002,"15min":2.4980000000000002},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":0.81999999999999995}]},{"osd":3,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":3.4100000000000001,"5min":3.4100000000000001,"15min":3.4100000000000001},"last":0.91800000000000004}]},{"osd":4,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":2.9649999999999999,"5min":2.9649999999999999,"15min":2.9649999999999999},"last":0.88}]},{"osd":5,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":3.476,"5min":3.476,"15min":3.476},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.764,"5min":1.764,"15min":1.764},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.90600000000000003}]},{"osd":6,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":4.6139999999999999,"5min":4.6139999999999999,"15min":4.6139999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"min":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"max":{"1min":4.5499999999999998,"5min":4.5499999999999998,"15min":4.5499999999999998},"last":0.99399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12780,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":12288,"kb_avail":94359060,"statfs":{"total":96636764160,"available":96623677440,"internally_reserved":0,"allocated":495616,"data_stored":217130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12582912},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.21},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.819}]},{"osd":1,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.423},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"min":{"1min":1.353,"5min":1.353,"15min":1.353},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.6819999999999999}]},{"osd":2,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":1.012,"5min":1.012,"15min":1.012},"max":{"1min":2.5510000000000002,"5min":2.5510000000000002,"15min":2.5510000000000002},"last":1.6379999999999999}]},{"osd":3,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":1.194,"5min":1.194,"15min":1.194},"max":{"1min":3.8559999999999999,"5min":3.8559999999999999,"15min":3.8559999999999999},"last":1.474},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"min":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.7190000000000001}]},{"osd":4,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"min":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"max":{"1min":6.4370000000000003,"5min":6.4370000000000003,"15min":6.4370000000000003},"last":1.3100000000000001}]},{"osd":5,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.724,"15min":1.724},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":3.5059999999999998,"5min":3.5059999999999998,"15min":3.5059999999999998},"last":1.238},{"interface":"front","average":{"1min":1.702,"5min":1.702,"15min":1.702},"min":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"max":{"1min":2.637,"5min":2.637,"15min":2.637},"last":1.532}]},{"osd":7,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"min":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"max":{"1min":3.6760000000000002,"5min":3.6760000000000002,"15min":3.6760000000000002},"last":1.768},{"interface":"front","average":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"min":{"1min":1.419,"5min":1.419,"15min":1.419},"max":{"1min":3.4940000000000002,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":1.5940000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12788,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":12288,"kb_avail":94359052,"statfs":{"total":96636764160,"available":96623669248,"internally_reserved":0,"allocated":503808,"data_stored":223517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12582912},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.22}]},{"osd":1,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.1220000000000001}]},{"osd":2,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.47},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.288}]},{"osd":3,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"min":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.802,"5min":1.802,"15min":1.802},"min":{"1min":1.194,"5min":1.194,"15min":1.194},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.589}]},{"osd":4,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"max":{"1min":2.4350000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.407},{"interface":"front","average":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"min":{"1min":1.157,"5min":1.157,"15min":1.157},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.7509999999999999}]},{"osd":6,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"min":{"1min":1.373,"5min":1.373,"15min":1.373},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.6779999999999999},{"interface":"front","average":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"min":{"1min":1.29,"5min":1.29,"15min":1.29},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":1.706}]},{"osd":7,"last update":"Fri Mar 3 06:25:59 2023","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.871,"15min":1.871},"min":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"max":{"1min":2.6539999999999999,"5min":2.6539999999999999,"15min":2.6539999999999999},"last":1.5549999999999999},{"interface":"front","average":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"min":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":1.8839999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12988,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":12480,"kb_avail":94358852,"statfs":{"total":96636764160,"available":96623464448,"internally_reserved":0,"allocated":512000,"data_stored":224995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12779520},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":3.887,"5min":3.887,"15min":3.887},"last":1.349},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.845,"5min":1.845,"15min":1.845},"last":1.319}]},{"osd":1,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.599},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":3.984,"5min":3.984,"15min":3.984},"last":1.105}]},{"osd":2,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.655,"5min":1.655,"15min":1.655},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":3.7669999999999999,"5min":3.7669999999999999,"15min":3.7669999999999999},"last":1.232}]},{"osd":3,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":1.071,"5min":1.071,"15min":1.071},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.2889999999999999}]},{"osd":5,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":1.881,"15min":1.881},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":7.0190000000000001,"5min":7.0190000000000001,"15min":7.0190000000000001},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"min":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":1.4219999999999999}]},{"osd":6,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.944,"5min":1.944,"15min":1.944},"min":{"1min":1.012,"5min":1.012,"15min":1.012},"max":{"1min":6.8810000000000002,"5min":6.8810000000000002,"15min":6.8810000000000002},"last":1.4590000000000001}]},{"osd":7,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"min":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"max":{"1min":3.5910000000000002,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.512},{"interface":"front","average":{"1min":1.748,"5min":1.748,"15min":1.748},"min":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.49}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12848,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":12352,"kb_avail":94358992,"statfs":{"total":96636764160,"available":96623607808,"internally_reserved":0,"allocated":499712,"data_stored":217096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12648448},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.7949999999999999,"5min":2.7949999999999999,"15min":2.7949999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.4670000000000001,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":0.88100000000000001}]},{"osd":1,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.94899999999999995}]},{"osd":2,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.014},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":1.3600000000000001}]},{"osd":4,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.569,"5min":2.569,"15min":2.569},"last":1.71},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":1.5369999999999999}]},{"osd":5,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.708,"5min":1.708,"15min":1.708},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":3.3999999999999999,"5min":3.3999999999999999,"15min":3.3999999999999999},"last":1.425},{"interface":"front","average":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"min":{"1min":1.333,"5min":1.333,"15min":1.333},"max":{"1min":2.9129999999999998,"5min":2.9129999999999998,"15min":2.9129999999999998},"last":1.4750000000000001}]},{"osd":6,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.823,"5min":1.823,"15min":1.823},"min":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"max":{"1min":2.8860000000000001,"5min":2.8860000000000001,"15min":2.8860000000000001},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.9670000000000001,"5min":2.9670000000000001,"15min":2.9670000000000001},"last":2.0630000000000002}]},{"osd":7,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.964},{"interface":"front","average":{"1min":1.722,"5min":1.722,"15min":1.722},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":3.3490000000000002,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.756}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8640,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363200,"statfs":{"total":96636764160,"available":96627916800,"internally_reserved":0,"allocated":516096,"data_stored":225017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.125}]},{"osd":1,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.427},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":0.94299999999999995}]},{"osd":3,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":1.28},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.224}]},{"osd":4,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.462},{"interface":"front","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":1.504}]},{"osd":5,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"min":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"max":{"1min":3.101,"5min":3.101,"15min":3.101},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.879,"5min":1.879,"15min":1.879},"min":{"1min":1.268,"5min":1.268,"15min":1.268},"max":{"1min":2.9100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":1.6950000000000001}]},{"osd":6,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"min":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.78},{"interface":"front","average":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.925}]},{"osd":7,"last update":"Fri Mar 3 06:26:01 2023","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"max":{"1min":3.0379999999999998,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":1.8979999999999999},{"interface":"front","average":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.829}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8744,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363096,"statfs":{"total":96636764160,"available":96627810304,"internally_reserved":0,"allocated":491520,"data_stored":216544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.879}]},{"osd":2,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.038},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":1.3340000000000001}]},{"osd":3,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.999,"5min":0.999,"15min":0.999},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":1.29}]},{"osd":4,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.383},{"interface":"front","average":{"1min":1.762,"5min":1.762,"15min":1.762},"min":{"1min":1.149,"5min":1.149,"15min":1.149},"max":{"1min":2.9399999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":1.5820000000000001}]},{"osd":5,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"min":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.641,"5min":1.641,"15min":1.641},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.6279999999999999}]},{"osd":6,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.764,"5min":1.764,"15min":1.764},"min":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.74}]},{"osd":7,"last update":"Fri Mar 3 06:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.845,"5min":1.845,"15min":1.845},"min":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.798,"5min":1.798,"15min":1.798},"min":{"1min":1.306,"5min":1.306,"15min":1.306},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.427}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13932,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94357908,"statfs":{"total":96636764160,"available":96622497792,"internally_reserved":0,"allocated":1085440,"data_stored":806876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.073},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.1359999999999999}]},{"osd":3,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.018},{"interface":"front","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.2190000000000001}]},{"osd":4,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":4.3419999999999996,"5min":4.3419999999999996,"15min":4.3419999999999996},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.766,"5min":1.766,"15min":1.766},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":3.0430000000000001,"5min":3.0430000000000001,"15min":3.0430000000000001},"last":1.651}]},{"osd":5,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":4.4779999999999998,"5min":4.4779999999999998,"15min":4.4779999999999998},"last":1.333},{"interface":"front","average":{"1min":1.841,"5min":1.841,"15min":1.841},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":4.415,"5min":4.415,"15min":4.415},"last":1.2829999999999999}]},{"osd":6,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":3.1899999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":6.3159999999999998,"5min":6.3159999999999998,"15min":6.3159999999999998},"last":1.498}]},{"osd":7,"last update":"Fri Mar 3 06:26:00 2023","interfaces":[{"interface":"back","average":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":5.8440000000000003,"5min":5.8440000000000003,"15min":5.8440000000000003},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.994,"5min":1.994,"15min":1.994},"min":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"max":{"1min":4.2619999999999996,"5min":4.2619999999999996,"15min":4.2619999999999996},"last":1.617}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-03T06:27:03.005 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-03-03T06:27:03.006 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-03-03T06:27:03.036 INFO:tasks.ceph.mds.b.smithi032.stderr:2023-03-03T06:27:03.032+0000 7fdbded2d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-03-03T06:27:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.258+0000 7f24949a4700 1 Processor -- start 2023-03-03T06:27:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.258+0000 7f24949a4700 1 -- start start 2023-03-03T06:27:03.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f24949a4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24900b45b0 0x7f2490160950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:03.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f24949a4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f24900d2290 0x7f249006ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:03.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f24949a4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2490160e90 con 0x7f24900b4980 2023-03-03T06:27:03.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f24949a4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24900d0df0 con 0x7f24900d2290 2023-03-03T06:27:03.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f24949a4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24900b8750 con 0x7f24900b45b0 2023-03-03T06:27:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f248ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24900b45b0 0x7f2490160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f248ed9d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24900b45b0 0x7f2490160950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:36594/0 (socket says 172.21.15.3:36594) 2023-03-03T06:27:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f248ed9d700 1 -- 172.21.15.3:0/2156168784 learned_addr learned my addr 172.21.15.3:0/2156168784 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f248f59e700 1 --2- 172.21.15.3:0/2156168784 >> v2:172.21.15.3:3300/0 conn(0x7f24900d2290 0x7f249006ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f248ed9d700 1 -- 172.21.15.3:0/2156168784 >> v2:172.21.15.3:3300/0 conn(0x7f24900d2290 msgr2=0x7f249006ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f248ed9d700 1 --2- 172.21.15.3:0/2156168784 >> v2:172.21.15.3:3300/0 conn(0x7f24900d2290 0x7f249006ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f248ed9d700 1 -- 172.21.15.3:0/2156168784 >> v1:172.21.15.3:6789/0 conn(0x7f24900b4980 legacy=0x7f249006e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f248ed9d700 1 -- 172.21.15.3:0/2156168784 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24900cd560 con 0x7f24900b45b0 2023-03-03T06:27:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.262+0000 7f248ed9d700 1 --2- 172.21.15.3:0/2156168784 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24900b45b0 0x7f2490160950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2480009910 tx=0x7f2480009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c3a12c260635254 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:03.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.266+0000 7f248e59c700 1 -- 172.21.15.3:0/2156168784 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480011830 con 0x7f24900b45b0 2023-03-03T06:27:03.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.266+0000 7f248e59c700 1 -- 172.21.15.3:0/2156168784 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2480011990 con 0x7f24900b45b0 2023-03-03T06:27:03.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.266+0000 7f248e59c700 1 -- 172.21.15.3:0/2156168784 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480011d40 con 0x7f24900b45b0 2023-03-03T06:27:03.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.266+0000 7f24949a4700 1 -- 172.21.15.3:0/2156168784 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24900b45b0 msgr2=0x7f2490160950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:03.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.266+0000 7f24949a4700 1 --2- 172.21.15.3:0/2156168784 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24900b45b0 0x7f2490160950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2480009910 tx=0x7f2480009c20 comp rx=0 tx=0).stop 2023-03-03T06:27:03.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.266+0000 7f24949a4700 1 -- 172.21.15.3:0/2156168784 shutdown_connections 2023-03-03T06:27:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.266+0000 7f24949a4700 1 --2- 172.21.15.3:0/2156168784 >> v2:172.21.15.3:3300/0 conn(0x7f24900d2290 0x7f249006ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.266+0000 7f24949a4700 1 --2- 172.21.15.3:0/2156168784 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24900b45b0 0x7f2490160950 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.266+0000 7f24949a4700 1 -- 172.21.15.3:0/2156168784 >> 172.21.15.3:0/2156168784 conn(0x7f2490071380 msgr2=0x7f24900cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.266+0000 7f24949a4700 1 -- 172.21.15.3:0/2156168784 shutdown_connections 2023-03-03T06:27:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.266+0000 7f24949a4700 1 -- 172.21.15.3:0/2156168784 wait complete. 2023-03-03T06:27:03.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f24949a4700 1 Processor -- start 2023-03-03T06:27:03.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f24949a4700 1 -- start start 2023-03-03T06:27:03.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f24949a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24900b45b0 0x7f24900cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:03.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f24949a4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24900b4980 0x7f249015d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f24949a4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24900d2290 0x7f249015f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f24949a4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249015def0 con 0x7f24900b4980 2023-03-03T06:27:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f24949a4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249015f750 con 0x7f24900b45b0 2023-03-03T06:27:03.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f24949a4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249015e170 con 0x7f24900d2290 2023-03-03T06:27:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f2487fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24900b4980 0x7f249015d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f2487fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24900b4980 0x7f249015d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33684/0 (socket says 172.21.15.3:33684) 2023-03-03T06:27:03.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f2487fff700 1 -- 172.21.15.3:0/2641367012 learned_addr learned my addr 172.21.15.3:0/2641367012 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f248ed9d700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24900b45b0 0x7f24900cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f248f59e700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24900d2290 0x7f249015f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f2487fff700 1 -- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24900d2290 msgr2=0x7f249015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f2487fff700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24900d2290 0x7f249015f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:03.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f2487fff700 1 -- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24900b45b0 msgr2=0x7f24900cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:03.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f2487fff700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24900b45b0 0x7f24900cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:03.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f2487fff700 1 -- 172.21.15.3:0/2641367012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2480009570 con 0x7f24900b4980 2023-03-03T06:27:03.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.270+0000 7f248f59e700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24900d2290 0x7f249015f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:27:03.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.274+0000 7f2487fff700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24900b4980 0x7f249015d9b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f247800c500 tx=0x7f247800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:03.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.274+0000 7f24877fe700 1 -- 172.21.15.3:0/2641367012 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24780052d0 con 0x7f24900b4980 2023-03-03T06:27:03.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.274+0000 7f24877fe700 1 -- 172.21.15.3:0/2641367012 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2478005430 con 0x7f24900b4980 2023-03-03T06:27:03.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.274+0000 7f24877fe700 1 -- 172.21.15.3:0/2641367012 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24780055a0 con 0x7f24900b4980 2023-03-03T06:27:03.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.274+0000 7f24949a4700 1 -- 172.21.15.3:0/2641367012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24900d07a0 con 0x7f24900b4980 2023-03-03T06:27:03.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.274+0000 7f24949a4700 1 -- 172.21.15.3:0/2641367012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24900cd840 con 0x7f24900b4980 2023-03-03T06:27:03.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.274+0000 7f24949a4700 1 -- 172.21.15.3:0/2641367012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24900d0a80 con 0x7f24900b4980 2023-03-03T06:27:03.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.278+0000 7f24877fe700 1 -- 172.21.15.3:0/2641367012 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f2478019030 con 0x7f24900b4980 2023-03-03T06:27:03.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.278+0000 7f24877fe700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2470064ee0 0x7f2470067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:03.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.278+0000 7f24877fe700 1 -- 172.21.15.3:0/2641367012 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2478081220 con 0x7f24900b4980 2023-03-03T06:27:03.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.278+0000 7f248ed9d700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2470064ee0 0x7f2470067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:03.286 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.282+0000 7f248ed9d700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2470064ee0 0x7f2470067390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2480014040 tx=0x7f2480008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:03.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.286+0000 7f24877fe700 1 -- 172.21.15.3:0/2641367012 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f247804f110 con 0x7f24900b4980 2023-03-03T06:27:03.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.518+0000 7f24949a4700 1 -- 172.21.15.3:0/2641367012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f2490160e90 con 0x7f24900b4980 2023-03-03T06:27:03.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.530+0000 7f24877fe700 1 -- 172.21.15.3:0/2641367012 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2490160e90 con 0x7f24900b4980 2023-03-03T06:27:03.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.534+0000 7f24949a4700 1 -- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2470064ee0 msgr2=0x7f2470067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:03.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.534+0000 7f24949a4700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2470064ee0 0x7f2470067390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2480014040 tx=0x7f2480008030 comp rx=0 tx=0).stop 2023-03-03T06:27:03.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.534+0000 7f24949a4700 1 -- 172.21.15.3:0/2641367012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24900b4980 msgr2=0x7f249015d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:03.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.534+0000 7f24949a4700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24900b4980 0x7f249015d9b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f247800c500 tx=0x7f247800c810 comp rx=0 tx=0).stop 2023-03-03T06:27:03.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.534+0000 7f24949a4700 1 -- 172.21.15.3:0/2641367012 shutdown_connections 2023-03-03T06:27:03.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.534+0000 7f24949a4700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2470064ee0 0x7f2470067390 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:03.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.534+0000 7f24949a4700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f24900d2290 0x7f249015f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:03.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.534+0000 7f24949a4700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24900b4980 0x7f249015d9b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:03.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.534+0000 7f24949a4700 1 --2- 172.21.15.3:0/2641367012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24900b45b0 0x7f24900cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:03.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.534+0000 7f24949a4700 1 -- 172.21.15.3:0/2641367012 >> 172.21.15.3:0/2641367012 conn(0x7f2490071380 msgr2=0x7f24900b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:03.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.534+0000 7f24949a4700 1 -- 172.21.15.3:0/2641367012 shutdown_connections 2023-03-03T06:27:03.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.534+0000 7f24949a4700 1 -- 172.21.15.3:0/2641367012 wait complete. 2023-03-03T06:27:03.545 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2023-03-03T06:27:03.577 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-03T06:27:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.778+0000 7fbb55f84700 1 Processor -- start 2023-03-03T06:27:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.778+0000 7fbb55f84700 1 -- start start 2023-03-03T06:27:03.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.782+0000 7fbb55f84700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbb500aed20 0x7fbb500cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:03.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.782+0000 7fbb55f84700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5015a5a0 con 0x7fbb500c9c10 2023-03-03T06:27:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.782+0000 7fbb55f84700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5015e230 con 0x7fbb5015dd30 2023-03-03T06:27:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.782+0000 7fbb55f84700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb500c97c0 con 0x7fbb500aed20 2023-03-03T06:27:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.782+0000 7fbb54f82700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbb500aed20 0x7fbb500cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.782+0000 7fbb55783700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbb500c9c10 0x7fbb500ca070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51712/0 (socket says 172.21.15.3:51712) 2023-03-03T06:27:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.782+0000 7fbb55783700 1 -- 172.21.15.3:0/4083530754 learned_addr learned my addr 172.21.15.3:0/4083530754 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:03.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb477fe700 1 -- 172.21.15.3:0/4083530754 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296474631 0 0) 0x7fbb5015a5a0 con 0x7fbb500c9c10 2023-03-03T06:27:03.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb477fe700 1 -- 172.21.15.3:0/4083530754 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb380035d0 con 0x7fbb500c9c10 2023-03-03T06:27:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb54f82700 1 -- 172.21.15.3:0/4083530754 >> v1:172.21.15.32:6789/0 conn(0x7fbb5015dd30 legacy=0x7fbb500af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb54f82700 1 -- 172.21.15.3:0/4083530754 >> v1:172.21.15.3:6789/0 conn(0x7fbb500c9c10 legacy=0x7fbb500ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb54f82700 1 -- 172.21.15.3:0/4083530754 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5015e510 con 0x7fbb500aed20 2023-03-03T06:27:03.791 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb54f82700 1 --2- 172.21.15.3:0/4083530754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb500aed20 0x7fbb500cb970 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009850 tx=0x7fbb4c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149387a5710f3d19 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb477fe700 1 -- 172.21.15.3:0/4083530754 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c011830 con 0x7fbb500aed20 2023-03-03T06:27:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb477fe700 1 -- 172.21.15.3:0/4083530754 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c011990 con 0x7fbb500aed20 2023-03-03T06:27:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb477fe700 1 -- 172.21.15.3:0/4083530754 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c011d40 con 0x7fbb500aed20 2023-03-03T06:27:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb55f84700 1 -- 172.21.15.3:0/4083530754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb500aed20 msgr2=0x7fbb500cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb55f84700 1 --2- 172.21.15.3:0/4083530754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb500aed20 0x7fbb500cb970 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009850 tx=0x7fbb4c009b60 comp rx=0 tx=0).stop 2023-03-03T06:27:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb55f84700 1 -- 172.21.15.3:0/4083530754 shutdown_connections 2023-03-03T06:27:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb55f84700 1 --2- 172.21.15.3:0/4083530754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb500aed20 0x7fbb500cb970 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb55f84700 1 -- 172.21.15.3:0/4083530754 >> 172.21.15.3:0/4083530754 conn(0x7fbb50070a30 msgr2=0x7fbb50070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb55f84700 1 -- 172.21.15.3:0/4083530754 shutdown_connections 2023-03-03T06:27:03.794 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.786+0000 7fbb55f84700 1 -- 172.21.15.3:0/4083530754 wait complete. 2023-03-03T06:27:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.790+0000 7fbb55f84700 1 Processor -- start 2023-03-03T06:27:03.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.790+0000 7fbb55f84700 1 -- start start 2023-03-03T06:27:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb55f84700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb500aed20 0x7fbb5015c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb55f84700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb500c9c10 0x7fbb500c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb55f84700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb5015dd30 0x7fbb50150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb55f84700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5015b1a0 con 0x7fbb500aed20 2023-03-03T06:27:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb55f84700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500c87f0 con 0x7fbb5015dd30 2023-03-03T06:27:03.799 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb55f84700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5015b3d0 con 0x7fbb500c9c10 2023-03-03T06:27:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb54f82700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb500aed20 0x7fbb5015c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb54f82700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb500aed20 0x7fbb5015c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33690/0 (socket says 172.21.15.3:33690) 2023-03-03T06:27:03.800 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb54f82700 1 -- 172.21.15.3:0/2248577905 learned_addr learned my addr 172.21.15.3:0/2248577905 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb55783700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb5015dd30 0x7fbb50150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb47fff700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb500c9c10 0x7fbb500c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb54f82700 1 -- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb500c9c10 msgr2=0x7fbb500c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb54f82700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb500c9c10 0x7fbb500c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb54f82700 1 -- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb5015dd30 msgr2=0x7fbb50150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb54f82700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb5015dd30 0x7fbb50150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:03.802 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb54f82700 1 -- 172.21.15.3:0/2248577905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c0094b0 con 0x7fbb500aed20 2023-03-03T06:27:03.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.794+0000 7fbb54f82700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb500aed20 0x7fbb5015c470 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c000c00 tx=0x7fbb4c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:03.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.798+0000 7fbb457fa700 1 -- 172.21.15.3:0/2248577905 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c01aa10 con 0x7fbb500aed20 2023-03-03T06:27:03.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.798+0000 7fbb457fa700 1 -- 172.21.15.3:0/2248577905 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c01f070 con 0x7fbb500aed20 2023-03-03T06:27:03.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.798+0000 7fbb457fa700 1 -- 172.21.15.3:0/2248577905 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c01adb0 con 0x7fbb500aed20 2023-03-03T06:27:03.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.798+0000 7fbb55f84700 1 -- 172.21.15.3:0/2248577905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5015ead0 con 0x7fbb500aed20 2023-03-03T06:27:03.803 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.798+0000 7fbb55f84700 1 -- 172.21.15.3:0/2248577905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5015e7f0 con 0x7fbb500aed20 2023-03-03T06:27:03.804 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.798+0000 7fbb55f84700 1 -- 172.21.15.3:0/2248577905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb500b42e0 con 0x7fbb500aed20 2023-03-03T06:27:03.806 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.802+0000 7fbb457fa700 1 -- 172.21.15.3:0/2248577905 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fbb4c00f050 con 0x7fbb500aed20 2023-03-03T06:27:03.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.802+0000 7fbb457fa700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbb3c064ee0 0x7fbb3c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:03.808 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.802+0000 7fbb457fa700 1 -- 172.21.15.3:0/2248577905 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbb4c083150 con 0x7fbb500aed20 2023-03-03T06:27:03.809 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.802+0000 7fbb47fff700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbb3c064ee0 0x7fbb3c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:03.810 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.806+0000 7fbb47fff700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbb3c064ee0 0x7fbb3c067390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbb500aea90 tx=0x7fbb40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:03.814 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:03.810+0000 7fbb457fa700 1 -- 172.21.15.3:0/2248577905 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fbb4c0510c0 con 0x7fbb500aed20 2023-03-03T06:27:04.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.026+0000 7fbb55f84700 1 -- 172.21.15.3:0/2248577905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb50073e90 con 0x7fbb500aed20 2023-03-03T06:27:04.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.026+0000 7fbb457fa700 1 -- 172.21.15.3:0/2248577905 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4643 (secure 0 0 0) 0x7fbb4c021090 con 0x7fbb500aed20 2023-03-03T06:27:04.031 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:27:04.032 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5358,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/3530112681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3530112681},{"type":"v1","addr":"172.21.15.3:6833","nonce":3530112681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2160506868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2160506868},{"type":"v1","addr":"172.21.15.3:6835","nonce":2160506868}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":54887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/4289059366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":4289059366},{"type":"v1","addr":"172.21.15.32:6836","nonce":4289059366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-03T06:26:57.500775+0000","modified":"2023-03-03T06:27:01.007679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":54890},"failed":[],"damaged":[],"stopped":[],"info":{"gid_54890":{"gid":54890,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.32:6837/947618180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":947618180},{"type":"v1","addr":"172.21.15.32:6837","nonce":947618180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-03-03T06:27:04.034 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.030+0000 7fbb55f84700 1 -- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbb3c064ee0 msgr2=0x7fbb3c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.030+0000 7fbb55f84700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbb3c064ee0 0x7fbb3c067390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbb500aea90 tx=0x7fbb40009040 comp rx=0 tx=0).stop 2023-03-03T06:27:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.030+0000 7fbb55f84700 1 -- 172.21.15.3:0/2248577905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb500aed20 msgr2=0x7fbb5015c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.030+0000 7fbb55f84700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb500aed20 0x7fbb5015c470 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c000c00 tx=0x7fbb4c004770 comp rx=0 tx=0).stop 2023-03-03T06:27:04.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.030+0000 7fbb55f84700 1 -- 172.21.15.3:0/2248577905 shutdown_connections 2023-03-03T06:27:04.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.030+0000 7fbb55f84700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fbb3c064ee0 0x7fbb3c067390 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.030+0000 7fbb55f84700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb5015dd30 0x7fbb50150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.030+0000 7fbb55f84700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fbb500c9c10 0x7fbb500c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.030+0000 7fbb55f84700 1 --2- 172.21.15.3:0/2248577905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb500aed20 0x7fbb5015c470 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.030+0000 7fbb55f84700 1 -- 172.21.15.3:0/2248577905 >> 172.21.15.3:0/2248577905 conn(0x7fbb50070a30 msgr2=0x7fbb50070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:04.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.030+0000 7fbb55f84700 1 -- 172.21.15.3:0/2248577905 shutdown_connections 2023-03-03T06:27:04.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.030+0000 7fbb55f84700 1 -- 172.21.15.3:0/2248577905 wait complete. 2023-03-03T06:27:04.040 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 17 2023-03-03T06:27:04.474 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-03-03T06:26:57.500775+0000', 'modified': '2023-03-03T06:27:01.007679+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 54890}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_54890': {'gid': 54890, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.32:6837/947618180', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 947618180}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 947618180}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-03-03T06:27:04.475 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-03-03T06:27:04.476 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-03-03T06:27:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.678+0000 7fadd97ac700 1 Processor -- start 2023-03-03T06:27:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.678+0000 7fadd97ac700 1 -- start start 2023-03-03T06:27:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.682+0000 7fadd97ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadd4158680 0x7fadd415c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.682+0000 7fadd97ac700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd40cae60 con 0x7fadd415c650 2023-03-03T06:27:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.682+0000 7fadd97ac700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd415cfd0 con 0x7fadd41582b0 2023-03-03T06:27:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.682+0000 7fadd97ac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd4158ad0 con 0x7fadd4158680 2023-03-03T06:27:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.682+0000 7fadd3fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fadd415c650 0x7fadd406e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51716/0 (socket says 172.21.15.3:51716) 2023-03-03T06:27:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.682+0000 7fadd3fff700 1 -- 172.21.15.3:0/1972200626 learned_addr learned my addr 172.21.15.3:0/1972200626 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.682+0000 7fadd27fc700 1 --2- 172.21.15.3:0/1972200626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadd4158680 0x7fadd415c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.682+0000 7fadd2ffd700 1 --1- 172.21.15.3:0/1972200626 >> v1:172.21.15.32:6789/0 conn(0x7fadd41582b0 0x7fadd40cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:54744/0 (socket says 172.21.15.3:54744) 2023-03-03T06:27:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.682+0000 7fadd1ffb700 1 -- 172.21.15.3:0/1972200626 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392342010 0 0) 0x7fadd40cae60 con 0x7fadd415c650 2023-03-03T06:27:04.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd1ffb700 1 -- 172.21.15.3:0/1972200626 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadbc0057e0 con 0x7fadd415c650 2023-03-03T06:27:04.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd27fc700 1 -- 172.21.15.3:0/1972200626 >> v1:172.21.15.32:6789/0 conn(0x7fadd41582b0 legacy=0x7fadd40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:04.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd1ffb700 1 -- 172.21.15.3:0/1972200626 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160062560 0 0) 0x7fadd415cfd0 con 0x7fadd41582b0 2023-03-03T06:27:04.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd27fc700 1 -- 172.21.15.3:0/1972200626 >> v1:172.21.15.3:6789/0 conn(0x7fadd415c650 legacy=0x7fadd406e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:04.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd27fc700 1 -- 172.21.15.3:0/1972200626 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd415ccf0 con 0x7fadd4158680 2023-03-03T06:27:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd27fc700 1 --2- 172.21.15.3:0/1972200626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadd4158680 0x7fadd415c110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009880 tx=0x7fadc8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cc4afa716bdec7e server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd1ffb700 1 -- 172.21.15.3:0/1972200626 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8011830 con 0x7fadd4158680 2023-03-03T06:27:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd1ffb700 1 -- 172.21.15.3:0/1972200626 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc8011990 con 0x7fadd4158680 2023-03-03T06:27:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd1ffb700 1 -- 172.21.15.3:0/1972200626 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8011d20 con 0x7fadd4158680 2023-03-03T06:27:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd97ac700 1 -- 172.21.15.3:0/1972200626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadd4158680 msgr2=0x7fadd415c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd97ac700 1 --2- 172.21.15.3:0/1972200626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadd4158680 0x7fadd415c110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009880 tx=0x7fadc8009b90 comp rx=0 tx=0).stop 2023-03-03T06:27:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd97ac700 1 -- 172.21.15.3:0/1972200626 shutdown_connections 2023-03-03T06:27:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd97ac700 1 --2- 172.21.15.3:0/1972200626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadd4158680 0x7fadd415c110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd97ac700 1 -- 172.21.15.3:0/1972200626 >> 172.21.15.3:0/1972200626 conn(0x7fadd406b580 msgr2=0x7fadd406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd97ac700 1 -- 172.21.15.3:0/1972200626 shutdown_connections 2023-03-03T06:27:04.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.686+0000 7fadd97ac700 1 -- 172.21.15.3:0/1972200626 wait complete. 2023-03-03T06:27:04.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.690+0000 7fadd97ac700 1 Processor -- start 2023-03-03T06:27:04.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.690+0000 7fadd97ac700 1 -- start start 2023-03-03T06:27:04.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd97ac700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadd41582b0 0x7fadd40b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd97ac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadd4158680 0x7fadd40c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd97ac700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd415c650 0x7fadd40c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd97ac700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40ca100 con 0x7fadd415c650 2023-03-03T06:27:04.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd97ac700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd414df90 con 0x7fadd4158680 2023-03-03T06:27:04.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd97ac700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd41e3040 con 0x7fadd41582b0 2023-03-03T06:27:04.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd415c650 0x7fadd40c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd2ffd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadd41582b0 0x7fadd40b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadd4158680 0x7fadd40c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd415c650 0x7fadd40c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33696/0 (socket says 172.21.15.3:33696) 2023-03-03T06:27:04.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd3fff700 1 -- 172.21.15.3:0/2523637451 learned_addr learned my addr 172.21.15.3:0/2523637451 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:04.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd3fff700 1 -- 172.21.15.3:0/2523637451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadd41582b0 msgr2=0x7fadd40b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:04.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd3fff700 1 --2- 172.21.15.3:0/2523637451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadd41582b0 0x7fadd40b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd3fff700 1 -- 172.21.15.3:0/2523637451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadd4158680 msgr2=0x7fadd40c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:04.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd3fff700 1 --2- 172.21.15.3:0/2523637451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadd4158680 0x7fadd40c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd3fff700 1 -- 172.21.15.3:0/2523637451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc80094b0 con 0x7fadd415c650 2023-03-03T06:27:04.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.694+0000 7fadd3fff700 1 --2- 172.21.15.3:0/2523637451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd415c650 0x7fadd40c8050 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00c400 tx=0x7fadcc00c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:04.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.698+0000 7fadbb7fe700 1 -- 172.21.15.3:0/2523637451 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc005b60 con 0x7fadd415c650 2023-03-03T06:27:04.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.698+0000 7fadbb7fe700 1 -- 172.21.15.3:0/2523637451 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadcc005cc0 con 0x7fadd415c650 2023-03-03T06:27:04.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.698+0000 7fadbb7fe700 1 -- 172.21.15.3:0/2523637451 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc007310 con 0x7fadd415c650 2023-03-03T06:27:04.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.698+0000 7fadd97ac700 1 -- 172.21.15.3:0/2523637451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd40cea80 con 0x7fadd415c650 2023-03-03T06:27:04.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.698+0000 7fadd97ac700 1 -- 172.21.15.3:0/2523637451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd40ce7a0 con 0x7fadd415c650 2023-03-03T06:27:04.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.702+0000 7fadbb7fe700 1 -- 172.21.15.3:0/2523637451 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fadcc019030 con 0x7fadd415c650 2023-03-03T06:27:04.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.702+0000 7fadbb7fe700 1 --2- 172.21.15.3:0/2523637451 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fadc0064f10 0x7fadc00673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7fadbb7fe700 1 -- 172.21.15.3:0/2523637451 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fadcc081310 con 0x7fadd415c650 2023-03-03T06:27:04.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7fadd2ffd700 1 --2- 172.21.15.3:0/2523637451 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fadc0064f10 0x7fadc00673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7faddca2b740 1 Processor -- start 2023-03-03T06:27:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7faddca2b740 1 -- start start 2023-03-03T06:27:04.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7fadd2ffd700 1 --2- 172.21.15.3:0/2523637451 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fadc0064f10 0x7fadc00673c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fadc40012e0 tx=0x7fadc4001b70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:04.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7faddca2b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 0xf8ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7faddca2b740 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3710 0xf8b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7faddca2b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 0xf3f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7faddca2b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0xe42f00 con 0xcd3ae0 2023-03-03T06:27:04.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7faddca2b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0xeb2ff0 con 0xd1e790 2023-03-03T06:27:04.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7faddca2b740 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0xeac2f0 con 0xcd3710 2023-03-03T06:27:04.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7fadd2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 0xf8ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7fadd2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 0xf8ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33706/0 (socket says 172.21.15.3:33706) 2023-03-03T06:27:04.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7fadd2ffd700 1 -- 172.21.15.3:0/2161698891 learned_addr learned my addr 172.21.15.3:0/2161698891 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:04.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7fadd27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3710 0xf8b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7fadd3fff700 1 --2- 172.21.15.3:0/2161698891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 0xf3f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7fadd3fff700 1 -- 172.21.15.3:0/2161698891 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3710 msgr2=0xf8b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:04.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7fadd3fff700 1 --2- 172.21.15.3:0/2161698891 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3710 0xf8b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7fadd3fff700 1 -- 172.21.15.3:0/2161698891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 msgr2=0xf8ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:04.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.706+0000 7fadd3fff700 1 --2- 172.21.15.3:0/2161698891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 0xf8ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7fadd3fff700 1 -- 172.21.15.3:0/2161698891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xed1c90 con 0xd1e790 2023-03-03T06:27:04.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7fadd2ffd700 1 --2- 172.21.15.3:0/2161698891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 0xf8ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:27:04.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7fadd3fff700 1 --2- 172.21.15.3:0/2161698891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 0xf3f340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fadcc052df0 tx=0x7fadcc005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef55c48a9261772b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:04.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7fadb97fa700 1 -- 172.21.15.3:0/2161698891 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc0810f0 con 0xd1e790 2023-03-03T06:27:04.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7fadb97fa700 1 -- 172.21.15.3:0/2161698891 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadcc051260 con 0xd1e790 2023-03-03T06:27:04.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7fadb97fa700 1 -- 172.21.15.3:0/2161698891 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc05ad00 con 0xd1e790 2023-03-03T06:27:04.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7faddca2b740 1 -- 172.21.15.3:0/2161698891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 msgr2=0xf3f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:04.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7faddca2b740 1 --2- 172.21.15.3:0/2161698891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 0xf3f340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fadcc052df0 tx=0x7fadcc005f00 comp rx=0 tx=0).stop 2023-03-03T06:27:04.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7faddca2b740 1 -- 172.21.15.3:0/2161698891 shutdown_connections 2023-03-03T06:27:04.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7faddca2b740 1 --2- 172.21.15.3:0/2161698891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 0xf3f340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7faddca2b740 1 --2- 172.21.15.3:0/2161698891 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3710 0xf8b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7faddca2b740 1 --2- 172.21.15.3:0/2161698891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 0xf8ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7faddca2b740 1 -- 172.21.15.3:0/2161698891 >> 172.21.15.3:0/2161698891 conn(0xcd3eb0 msgr2=0xf6c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:04.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7faddca2b740 1 -- 172.21.15.3:0/2161698891 shutdown_connections 2023-03-03T06:27:04.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.710+0000 7faddca2b740 1 -- 172.21.15.3:0/2161698891 wait complete. 2023-03-03T06:27:04.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7faddca2b740 1 -- start start 2023-03-03T06:27:04.719 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7faddca2b740 1 Processor -- start 2023-03-03T06:27:04.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7faddca2b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3710 0xf55720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7faddca2b740 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3ae0 0xf51d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7faddca2b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 0xf522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7faddca2b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0xd58b20 con 0xcd3710 2023-03-03T06:27:04.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7faddca2b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0xf52c50 con 0xd1e790 2023-03-03T06:27:04.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7faddca2b740 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0xf69ac0 con 0xcd3ae0 2023-03-03T06:27:04.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7faddca2b740 10 client.0 _ll_register_callbacks cb 0xf07d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-03T06:27:04.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7fadd2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3710 0xf55720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7fadd2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3710 0xf55720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33716/0 (socket says 172.21.15.3:33716) 2023-03-03T06:27:04.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7fadd2ffd700 1 -- 172.21.15.3:0/3626286374 learned_addr learned my addr 172.21.15.3:0/3626286374 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:04.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7fadd3fff700 1 --2- 172.21.15.3:0/3626286374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 0xf522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7fadd27fc700 1 --2- 172.21.15.3:0/3626286374 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3ae0 0xf51d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7fadd2ffd700 1 -- 172.21.15.3:0/3626286374 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3ae0 msgr2=0xf51d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:04.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7fadd2ffd700 1 --2- 172.21.15.3:0/3626286374 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3ae0 0xf51d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7fadd2ffd700 1 -- 172.21.15.3:0/3626286374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 msgr2=0xf522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:04.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7fadd2ffd700 1 --2- 172.21.15.3:0/3626286374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 0xf522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7fadd2ffd700 1 -- 172.21.15.3:0/3626286374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd40ce7a0 con 0xcd3710 2023-03-03T06:27:04.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7fadd2ffd700 1 --2- 172.21.15.3:0/3626286374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3710 0xf55720 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fadc40123f0 tx=0x7fadc4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b9ec4d89acc193 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:04.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.714+0000 7fadb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-03-03T06:27:04.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7fadb1ffb700 1 -- 172.21.15.3:0/3626286374 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc4013540 con 0xcd3710 2023-03-03T06:27:04.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7fadb1ffb700 1 -- 172.21.15.3:0/3626286374 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fadc401b070 con 0xcd3710 2023-03-03T06:27:04.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7faddca2b740 1 -- 172.21.15.3:0/3626286374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xea5370 con 0xcd3710 2023-03-03T06:27:04.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7fadb1ffb700 1 -- 172.21.15.3:0/3626286374 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadc4012c20 con 0xcd3710 2023-03-03T06:27:04.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7faddca2b740 10 client.5541 fetch_fsmap learned FSMap version 17 2023-03-03T06:27:04.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7faddca2b740 1 -- 172.21.15.3:0/3626286374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0xd1f9e0 con 0xcd3710 2023-03-03T06:27:04.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7fadb1ffb700 1 -- 172.21.15.3:0/3626286374 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xea5370 con 0xcd3710 2023-03-03T06:27:04.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7fadb1ffb700 1 -- 172.21.15.3:0/3626286374 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 17) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fadc4029350 con 0xcd3710 2023-03-03T06:27:04.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7faddca2b740 10 client.5541 fetch_fsmap finished waiting for FSMap version 17 2023-03-03T06:27:04.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7faddca2b740 10 client.5541 resolve_mds: resolved name 'b' to daemon mds.b 2023-03-03T06:27:04.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7faddca2b740 20 client.5541 populate_metadata read hostname 'smithi003' 2023-03-03T06:27:04.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7faddca2b740 1 --2- 172.21.15.3:0/3626286374 >> [v2:172.21.15.32:6835/947618180,v1:172.21.15.32:6837/947618180] conn(0xf54380 0xf509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7faddca2b740 4 client.5541 mds_command: new command op to 54890 tid=0[{"prefix": "get_command_descriptions"}] 2023-03-03T06:27:04.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7faddca2b740 1 -- 172.21.15.3:0/3626286374 --> [v2:172.21.15.32:6835/947618180,v1:172.21.15.32:6837/947618180] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xea5090 con 0xf54380 2023-03-03T06:27:04.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.718+0000 7fadd27fc700 1 --2- 172.21.15.3:0/3626286374 >> [v2:172.21.15.32:6835/947618180,v1:172.21.15.32:6837/947618180] conn(0xf54380 0xf509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7fadd27fc700 1 --2- 172.21.15.3:0/3626286374 >> [v2:172.21.15.32:6835/947618180,v1:172.21.15.32:6837/947618180] conn(0xf54380 0xf509b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e9b04e92c53490b server_cookie=7f5bbda84293d717 in_seq=0 out_seq=0 2023-03-03T06:27:04.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7fadb1ffb700 10 client.5541 ms_handle_connect on v2:172.21.15.32:6835/947618180 2023-03-03T06:27:04.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7fadb1ffb700 1 -- 172.21.15.3:0/3626286374 <== mds.0 v2:172.21.15.32:6835/947618180 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18278 (crc 0 0 0) 0xea5090 con 0xf54380 2023-03-03T06:27:04.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7fadb1ffb700 10 client.5541 handle_command_reply: tid=0 2023-03-03T06:27:04.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7faddca2b740 1 client.5541 shutdown 2023-03-03T06:27:04.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7faddca2b740 1 -- 172.21.15.3:0/3626286374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3710 msgr2=0xf55720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:04.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7faddca2b740 1 --2- 172.21.15.3:0/3626286374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3710 0xf55720 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fadc40123f0 tx=0x7fadc4001250 comp rx=0 tx=0).stop 2023-03-03T06:27:04.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7faddca2b740 1 -- 172.21.15.3:0/3626286374 shutdown_connections 2023-03-03T06:27:04.729 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7faddca2b740 1 --2- 172.21.15.3:0/3626286374 >> [v2:172.21.15.32:6835/947618180,v1:172.21.15.32:6837/947618180] conn(0xf54380 0xf509b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7faddca2b740 1 --2- 172.21.15.3:0/3626286374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 0xf522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7faddca2b740 1 --2- 172.21.15.3:0/3626286374 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3ae0 0xf51d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7faddca2b740 1 --2- 172.21.15.3:0/3626286374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3710 0xf55720 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7faddca2b740 1 -- 172.21.15.3:0/3626286374 >> 172.21.15.3:0/3626286374 conn(0xcd3eb0 msgr2=0xf6bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:04.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7fadb1ffb700 0 client.5541 ms_handle_reset on v2:172.21.15.32:6835/947618180 2023-03-03T06:27:04.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7faddca2b740 1 -- 172.21.15.3:0/3626286374 shutdown_connections 2023-03-03T06:27:04.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.722+0000 7faddca2b740 1 -- 172.21.15.3:0/3626286374 wait complete. 2023-03-03T06:27:04.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.726+0000 7faddca2b740 20 client.5541 trim_cache size 0 max 16384 2023-03-03T06:27:04.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.742+0000 7faddca2b740 1 Processor -- start 2023-03-03T06:27:04.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.742+0000 7faddca2b740 1 -- start start 2023-03-03T06:27:04.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7faddca2b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3710 0xf52290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7faddca2b740 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3ae0 0xf88d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7faddca2b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 0xf89260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7faddca2b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0xeaee50 con 0xcd3710 2023-03-03T06:27:04.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7faddca2b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0xf4f6f0 con 0xd1e790 2023-03-03T06:27:04.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7faddca2b740 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0xea7c60 con 0xcd3ae0 2023-03-03T06:27:04.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadd2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3710 0xf52290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadd27fc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3ae0 0xf88d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadd2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3710 0xf52290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33726/0 (socket says 172.21.15.3:33726) 2023-03-03T06:27:04.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadd2ffd700 1 -- 172.21.15.3:0/3028603325 learned_addr learned my addr 172.21.15.3:0/3028603325 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:04.752 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadd3fff700 1 --2- 172.21.15.3:0/3028603325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 0xf89260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadd27fc700 1 -- 172.21.15.3:0/3028603325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 msgr2=0xf89260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:04.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadd27fc700 1 --2- 172.21.15.3:0/3028603325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 0xf89260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadd27fc700 1 -- 172.21.15.3:0/3028603325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3710 msgr2=0xf52290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:04.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadd27fc700 1 --2- 172.21.15.3:0/3028603325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3710 0xf52290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadd27fc700 1 -- 172.21.15.3:0/3028603325 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd414ec60 con 0xcd3ae0 2023-03-03T06:27:04.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadd27fc700 1 --2- 172.21.15.3:0/3028603325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3ae0 0xf88d20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fadc8015000 tx=0x7fadc8010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=277aa7762d2675e7 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:04.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadb9ffb700 1 -- 172.21.15.3:0/3028603325 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8011e40 con 0xcd3ae0 2023-03-03T06:27:04.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadb9ffb700 1 -- 172.21.15.3:0/3028603325 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc802ab70 con 0xcd3ae0 2023-03-03T06:27:04.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.746+0000 7fadb9ffb700 1 -- 172.21.15.3:0/3028603325 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc802a120 con 0xcd3ae0 2023-03-03T06:27:04.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 -- 172.21.15.3:0/3028603325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3ae0 msgr2=0xf88d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:04.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 --2- 172.21.15.3:0/3028603325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3ae0 0xf88d20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fadc8015000 tx=0x7fadc8010000 comp rx=0 tx=0).stop 2023-03-03T06:27:04.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 -- 172.21.15.3:0/3028603325 shutdown_connections 2023-03-03T06:27:04.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 --2- 172.21.15.3:0/3028603325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xd1e790 0xf89260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 --2- 172.21.15.3:0/3028603325 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xcd3ae0 0xf88d20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 --2- 172.21.15.3:0/3028603325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3710 0xf52290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 -- 172.21.15.3:0/3028603325 >> 172.21.15.3:0/3028603325 conn(0xcd3eb0 msgr2=0xf6bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:04.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 -- 172.21.15.3:0/3028603325 shutdown_connections 2023-03-03T06:27:04.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 -- 172.21.15.3:0/3028603325 wait complete. 2023-03-03T06:27:04.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 -- start start 2023-03-03T06:27:04.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 Processor -- start 2023-03-03T06:27:04.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xcd3710 0xff2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 0xf89f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xd1e790 0xf8a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0xe7d2e0 con 0xcd3ae0 2023-03-03T06:27:04.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0xf6cb70 con 0xcd3710 2023-03-03T06:27:04.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0xea9b00 con 0xd1e790 2023-03-03T06:27:04.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7faddca2b740 10 client.0 _ll_register_callbacks cb 0xeab080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-03T06:27:04.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7fadd27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 0xf89f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7fadd27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 0xf89f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33732/0 (socket says 172.21.15.3:33732) 2023-03-03T06:27:04.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.750+0000 7fadd27fc700 1 -- 172.21.15.3:0/3980348268 learned_addr learned my addr 172.21.15.3:0/3980348268 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:04.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadd2ffd700 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xcd3710 0xff2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadd3fff700 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xd1e790 0xf8a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadd27fc700 1 -- 172.21.15.3:0/3980348268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xd1e790 msgr2=0xf8a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:04.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadd27fc700 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xd1e790 0xf8a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadd27fc700 1 -- 172.21.15.3:0/3980348268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xcd3710 msgr2=0xff2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:04.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadd27fc700 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xcd3710 0xff2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadd27fc700 1 -- 172.21.15.3:0/3980348268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd415ccf0 con 0xcd3ae0 2023-03-03T06:27:04.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadd2ffd700 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xcd3710 0xff2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:27:04.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadd27fc700 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 0xf89f90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fadc80090b0 tx=0x7fadc80047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd56413d1e89dd1 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:04.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadb1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2023-03-03T06:27:04.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadb1ffb700 1 -- 172.21.15.3:0/3980348268 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc802a320 con 0xcd3ae0 2023-03-03T06:27:04.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadb1ffb700 1 -- 172.21.15.3:0/3980348268 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fadc801f040 con 0xcd3ae0 2023-03-03T06:27:04.761 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7faddca2b740 1 -- 172.21.15.3:0/3980348268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadcc051260 con 0xcd3ae0 2023-03-03T06:27:04.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadb1ffb700 1 -- 172.21.15.3:0/3980348268 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadc8011e50 con 0xcd3ae0 2023-03-03T06:27:04.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7faddca2b740 10 client.5547 fetch_fsmap learned FSMap version 17 2023-03-03T06:27:04.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7faddca2b740 1 -- 172.21.15.3:0/3980348268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0xe58e40 con 0xcd3ae0 2023-03-03T06:27:04.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadb1ffb700 1 -- 172.21.15.3:0/3980348268 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadcc051260 con 0xcd3ae0 2023-03-03T06:27:04.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadb1ffb700 1 -- 172.21.15.3:0/3980348268 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 17) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fadc802a480 con 0xcd3ae0 2023-03-03T06:27:04.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7faddca2b740 10 client.5547 fetch_fsmap finished waiting for FSMap version 17 2023-03-03T06:27:04.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7faddca2b740 10 client.5547 resolve_mds: resolved name 'b' to daemon mds.b 2023-03-03T06:27:04.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7faddca2b740 20 client.5547 populate_metadata read hostname 'smithi003' 2023-03-03T06:27:04.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7faddca2b740 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.32:6835/947618180,v1:172.21.15.32:6837/947618180] conn(0xf41c50 0xf42020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:04.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7faddca2b740 4 client.5547 mds_command: new command op to 54890 tid=0[{"prefix": "status"}] 2023-03-03T06:27:04.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7faddca2b740 1 -- 172.21.15.3:0/3980348268 --> [v2:172.21.15.32:6835/947618180,v1:172.21.15.32:6837/947618180] -- command(tid 0: {"prefix": "status"}) v1 -- 0xd50260 con 0xf41c50 2023-03-03T06:27:04.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.754+0000 7fadd2ffd700 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.32:6835/947618180,v1:172.21.15.32:6837/947618180] conn(0xf41c50 0xf42020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:04.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7fadd2ffd700 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.32:6835/947618180,v1:172.21.15.32:6837/947618180] conn(0xf41c50 0xf42020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55164c2e816f2ed1 server_cookie=3ec0c24cc72654ed in_seq=0 out_seq=0 2023-03-03T06:27:04.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7fadb1ffb700 10 client.5547 ms_handle_connect on v2:172.21.15.32:6835/947618180 2023-03-03T06:27:04.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7fadb1ffb700 1 -- 172.21.15.3:0/3980348268 <== mds.0 v2:172.21.15.32:6835/947618180 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0xd50260 con 0xf41c50 2023-03-03T06:27:04.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7fadb1ffb700 10 client.5547 handle_command_reply: tid=0 2023-03-03T06:27:04.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7faddca2b740 1 client.5547 shutdown 2023-03-03T06:27:04.765 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7faddca2b740 1 -- 172.21.15.3:0/3980348268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 msgr2=0xf89f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7faddca2b740 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 0xf89f90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fadc80090b0 tx=0x7fadc80047e0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7faddca2b740 1 -- 172.21.15.3:0/3980348268 shutdown_connections 2023-03-03T06:27:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7faddca2b740 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.32:6835/947618180,v1:172.21.15.32:6837/947618180] conn(0xf41c50 0xf42020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7faddca2b740 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0xd1e790 0xf8a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7faddca2b740 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcd3ae0 0xf89f90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.766 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7faddca2b740 1 --2- 172.21.15.3:0/3980348268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0xcd3710 0xff2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7faddca2b740 1 -- 172.21.15.3:0/3980348268 >> 172.21.15.3:0/3980348268 conn(0xcd3eb0 msgr2=0xf4e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:04.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7fadb1ffb700 0 client.5547 ms_handle_reset on v2:172.21.15.32:6835/947618180 2023-03-03T06:27:04.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7faddca2b740 1 -- 172.21.15.3:0/3980348268 shutdown_connections 2023-03-03T06:27:04.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7faddca2b740 1 -- 172.21.15.3:0/3980348268 wait complete. 2023-03-03T06:27:04.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.758+0000 7faddca2b740 20 client.5547 trim_cache size 0 max 16384 2023-03-03T06:27:04.767 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2023-03-03T06:27:04.768 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "7d496254-7268-405f-a1d8-c8033dc8659a", 2023-03-03T06:27:04.768 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2023-03-03T06:27:04.768 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 54890, 2023-03-03T06:27:04.768 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2023-03-03T06:27:04.768 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2023-03-03T06:27:04.768 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2023-03-03T06:27:04.768 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 7.2519203179999998, 2023-03-03T06:27:04.768 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 17, 2023-03-03T06:27:04.769 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 43, 2023-03-03T06:27:04.769 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 43, 2023-03-03T06:27:04.769 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 22.469885826999999 2023-03-03T06:27:04.769 INFO:teuthology.orchestra.run.smithi003.stdout:} 2023-03-03T06:27:04.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.762+0000 7fadd97ac700 1 -- 172.21.15.3:0/2523637451 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fadc0064f10 msgr2=0x7fadc00673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:04.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.762+0000 7fadd97ac700 1 --2- 172.21.15.3:0/2523637451 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fadc0064f10 0x7fadc00673c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fadc40012e0 tx=0x7fadc4001b70 comp rx=0 tx=0).stop 2023-03-03T06:27:04.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.762+0000 7fadd97ac700 1 -- 172.21.15.3:0/2523637451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd415c650 msgr2=0x7fadd40c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:04.770 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.762+0000 7fadd97ac700 1 --2- 172.21.15.3:0/2523637451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd415c650 0x7fadd40c8050 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00c400 tx=0x7fadcc00c710 comp rx=0 tx=0).stop 2023-03-03T06:27:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.762+0000 7fadd97ac700 1 -- 172.21.15.3:0/2523637451 shutdown_connections 2023-03-03T06:27:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.762+0000 7fadd97ac700 1 --2- 172.21.15.3:0/2523637451 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fadc0064f10 0x7fadc00673c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.762+0000 7fadd97ac700 1 --2- 172.21.15.3:0/2523637451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadd415c650 0x7fadd40c8050 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.762+0000 7fadd97ac700 1 --2- 172.21.15.3:0/2523637451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fadd4158680 0x7fadd40c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.762+0000 7fadd97ac700 1 --2- 172.21.15.3:0/2523637451 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fadd41582b0 0x7fadd40b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.762+0000 7fadd97ac700 1 -- 172.21.15.3:0/2523637451 >> 172.21.15.3:0/2523637451 conn(0x7fadd406b580 msgr2=0x7fadd406e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:04.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.762+0000 7fadd97ac700 1 -- 172.21.15.3:0/2523637451 shutdown_connections 2023-03-03T06:27:04.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.762+0000 7fadd97ac700 1 -- 172.21.15.3:0/2523637451 wait complete. 2023-03-03T06:27:04.800 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-03-03T06:27:05.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.998+0000 7f60c1e5a700 1 Processor -- start 2023-03-03T06:27:05.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:04.998+0000 7f60c1e5a700 1 -- start start 2023-03-03T06:27:05.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60c1e5a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc153670 0x7f60bc071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60c1e5a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60bc1608d0 0x7f60bc0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60c1e5a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60bc160440 con 0x7f60bc160d20 2023-03-03T06:27:05.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60c1e5a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60bc0cd3e0 con 0x7f60bc1608d0 2023-03-03T06:27:05.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60c1e5a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc0cd6c0 con 0x7f60bc153670 2023-03-03T06:27:05.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60c0e58700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc153670 0x7f60bc071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60c1659700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f60bc160d20 0x7f60bc0d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51718/0 (socket says 172.21.15.3:51718) 2023-03-03T06:27:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60c1659700 1 -- 172.21.15.3:0/3006354164 learned_addr learned my addr 172.21.15.3:0/3006354164 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60bbfff700 1 --2- 172.21.15.3:0/3006354164 >> v2:172.21.15.3:3300/0 conn(0x7f60bc1608d0 0x7f60bc0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60c0e58700 1 -- 172.21.15.3:0/3006354164 >> v2:172.21.15.3:3300/0 conn(0x7f60bc1608d0 msgr2=0x7f60bc0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60c0e58700 1 --2- 172.21.15.3:0/3006354164 >> v2:172.21.15.3:3300/0 conn(0x7f60bc1608d0 0x7f60bc0cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:05.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60c0e58700 1 -- 172.21.15.3:0/3006354164 >> v1:172.21.15.3:6789/0 conn(0x7f60bc160d20 legacy=0x7f60bc0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:05.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60c0e58700 1 -- 172.21.15.3:0/3006354164 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60bc1600d0 con 0x7f60bc153670 2023-03-03T06:27:05.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60bb7fe700 1 -- 172.21.15.3:0/3006354164 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288233653 0 0) 0x7f60bc160440 con 0x7f60bc160d20 2023-03-03T06:27:05.009 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60c0e58700 1 --2- 172.21.15.3:0/3006354164 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc153670 0x7f60bc071350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009850 tx=0x7f60b0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eccca4d3ab83a4ab server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:05.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60bb7fe700 1 -- 172.21.15.3:0/3006354164 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0011830 con 0x7f60bc153670 2023-03-03T06:27:05.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60bb7fe700 1 -- 172.21.15.3:0/3006354164 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b0011990 con 0x7f60bc153670 2023-03-03T06:27:05.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.002+0000 7f60bb7fe700 1 -- 172.21.15.3:0/3006354164 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0011d40 con 0x7f60bc153670 2023-03-03T06:27:05.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.006+0000 7f60c1e5a700 1 -- 172.21.15.3:0/3006354164 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc153670 msgr2=0x7f60bc071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:05.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.006+0000 7f60c1e5a700 1 --2- 172.21.15.3:0/3006354164 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc153670 0x7f60bc071350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009850 tx=0x7f60b0009b60 comp rx=0 tx=0).stop 2023-03-03T06:27:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.006+0000 7f60c1e5a700 1 -- 172.21.15.3:0/3006354164 shutdown_connections 2023-03-03T06:27:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.006+0000 7f60c1e5a700 1 --2- 172.21.15.3:0/3006354164 >> v2:172.21.15.3:3300/0 conn(0x7f60bc1608d0 0x7f60bc0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.006+0000 7f60c1e5a700 1 --2- 172.21.15.3:0/3006354164 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc153670 0x7f60bc071350 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.006+0000 7f60c1e5a700 1 -- 172.21.15.3:0/3006354164 >> 172.21.15.3:0/3006354164 conn(0x7f60bc06e5c0 msgr2=0x7f60bc06e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:05.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.006+0000 7f60c1e5a700 1 -- 172.21.15.3:0/3006354164 shutdown_connections 2023-03-03T06:27:05.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.006+0000 7f60c1e5a700 1 -- 172.21.15.3:0/3006354164 wait complete. 2023-03-03T06:27:05.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.006+0000 7f60c1e5a700 1 Processor -- start 2023-03-03T06:27:05.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.006+0000 7f60c1e5a700 1 -- start start 2023-03-03T06:27:05.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60c1e5a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60bc153670 0x7f60bc0b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60c1e5a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc1608d0 0x7f60bc155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60c1e5a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60bc160d20 0x7f60bc14ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60c1e5a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc0b3c70 con 0x7f60bc160d20 2023-03-03T06:27:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60c1e5a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc0b3f50 con 0x7f60bc153670 2023-03-03T06:27:05.016 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60c1e5a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc0b4230 con 0x7f60bc1608d0 2023-03-03T06:27:05.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60c1659700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60bc160d20 0x7f60bc14ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60c0e58700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60bc153670 0x7f60bc0b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60bbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc1608d0 0x7f60bc155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60bbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc1608d0 0x7f60bc155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:36688/0 (socket says 172.21.15.3:36688) 2023-03-03T06:27:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60c0e58700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60bc153670 0x7f60bc0b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:37462/0 (socket says 172.21.15.3:37462) 2023-03-03T06:27:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60bbfff700 1 -- 172.21.15.3:0/928491428 learned_addr learned my addr 172.21.15.3:0/928491428 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:05.018 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60c1659700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60bc160d20 0x7f60bc14ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33752/0 (socket says 172.21.15.3:33752) 2023-03-03T06:27:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60bbfff700 1 -- 172.21.15.3:0/928491428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60bc153670 msgr2=0x7f60bc0b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60bbfff700 1 --2- 172.21.15.3:0/928491428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60bc153670 0x7f60bc0b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60bbfff700 1 -- 172.21.15.3:0/928491428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60bc160d20 msgr2=0x7f60bc14ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60bbfff700 1 --2- 172.21.15.3:0/928491428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60bc160d20 0x7f60bc14ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.010+0000 7f60bbfff700 1 -- 172.21.15.3:0/928491428 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b00094b0 con 0x7f60bc1608d0 2023-03-03T06:27:05.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.014+0000 7f60bbfff700 1 --2- 172.21.15.3:0/928491428 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc1608d0 0x7f60bc155cc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f60a800e9d0 tx=0x7f60a800ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.014+0000 7f60b97fa700 1 -- 172.21.15.3:0/928491428 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8009720 con 0x7f60bc1608d0 2023-03-03T06:27:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.014+0000 7f60b97fa700 1 -- 172.21.15.3:0/928491428 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a800ee70 con 0x7f60bc1608d0 2023-03-03T06:27:05.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.014+0000 7f60c1e5a700 1 -- 172.21.15.3:0/928491428 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60bc0cd9a0 con 0x7f60bc1608d0 2023-03-03T06:27:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.014+0000 7f60b97fa700 1 -- 172.21.15.3:0/928491428 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a800ca50 con 0x7f60bc1608d0 2023-03-03T06:27:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.014+0000 7f60c1e5a700 1 -- 172.21.15.3:0/928491428 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60bc071890 con 0x7f60bc1608d0 2023-03-03T06:27:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.014+0000 7f60c1e5a700 1 -- 172.21.15.3:0/928491428 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60bc0cdb30 con 0x7f60bc1608d0 2023-03-03T06:27:05.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.018+0000 7f60b97fa700 1 -- 172.21.15.3:0/928491428 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f60a8019030 con 0x7f60bc1608d0 2023-03-03T06:27:05.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.018+0000 7f60b97fa700 1 --2- 172.21.15.3:0/928491428 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f60a0064ee0 0x7f60a0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.018+0000 7f60b97fa700 1 -- 172.21.15.3:0/928491428 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f60a8081f70 con 0x7f60bc1608d0 2023-03-03T06:27:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.018+0000 7f60c0e58700 1 --2- 172.21.15.3:0/928491428 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f60a0064ee0 0x7f60a0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:05.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.018+0000 7f60c0e58700 1 --2- 172.21.15.3:0/928491428 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f60a0064ee0 0x7f60a0067390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f60b0000c00 tx=0x7f60b0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:05.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.026+0000 7f60b97fa700 1 -- 172.21.15.3:0/928491428 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f60a804fee0 con 0x7f60bc1608d0 2023-03-03T06:27:05.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.242+0000 7f60c1e5a700 1 -- 172.21.15.3:0/928491428 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f60bc0b4940 con 0x7f60bc1608d0 2023-03-03T06:27:05.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.246+0000 7f60b97fa700 1 -- 172.21.15.3:0/928491428 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f60a8053500 con 0x7f60bc1608d0 2023-03-03T06:27:05.250 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-03-03T06:27:05.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.246+0000 7f60c1e5a700 1 -- 172.21.15.3:0/928491428 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f60a0064ee0 msgr2=0x7f60a0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.246+0000 7f60c1e5a700 1 --2- 172.21.15.3:0/928491428 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f60a0064ee0 0x7f60a0067390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f60b0000c00 tx=0x7f60b0008030 comp rx=0 tx=0).stop 2023-03-03T06:27:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.246+0000 7f60c1e5a700 1 -- 172.21.15.3:0/928491428 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc1608d0 msgr2=0x7f60bc155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:05.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.246+0000 7f60c1e5a700 1 --2- 172.21.15.3:0/928491428 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc1608d0 0x7f60bc155cc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f60a800e9d0 tx=0x7f60a800ece0 comp rx=0 tx=0).stop 2023-03-03T06:27:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.250+0000 7f60c1e5a700 1 -- 172.21.15.3:0/928491428 shutdown_connections 2023-03-03T06:27:05.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.250+0000 7f60c1e5a700 1 --2- 172.21.15.3:0/928491428 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f60a0064ee0 0x7f60a0067390 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.250+0000 7f60c1e5a700 1 --2- 172.21.15.3:0/928491428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60bc160d20 0x7f60bc14ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.250+0000 7f60c1e5a700 1 --2- 172.21.15.3:0/928491428 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f60bc1608d0 0x7f60bc155cc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.250+0000 7f60c1e5a700 1 --2- 172.21.15.3:0/928491428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60bc153670 0x7f60bc0b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:05.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.250+0000 7f60c1e5a700 1 -- 172.21.15.3:0/928491428 >> 172.21.15.3:0/928491428 conn(0x7f60bc06e5c0 msgr2=0x7f60bc0cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.250+0000 7f60c1e5a700 1 -- 172.21.15.3:0/928491428 shutdown_connections 2023-03-03T06:27:05.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:05.250+0000 7f60c1e5a700 1 -- 172.21.15.3:0/928491428 wait complete. 2023-03-03T06:27:05.289 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-03-03T06:27:05.289 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-03-03T06:27:05.290 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2023-03-03T06:27:05.290 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-03-03T06:27:05.290 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-03-03T06:27:05.291 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-03-03T06:27:05.291 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-03-03T06:27:05.291 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-03-03T06:27:05.291 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-03-03T06:27:05.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:05.292 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2023-03-03T06:27:05.314 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-03T06:27:05.314 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-03T06:27:05.315 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2023-03-03T06:27:05.315 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:27:05.315 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2023-03-03T06:27:05.315 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:27:05.315 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:27:05.315 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2023-03-03T06:27:05.315 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2023-03-03T06:27:05.316 INFO:teuthology.orchestra.run.smithi003.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-03-03T06:27:05.316 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2023-03-03T06:27:05.316 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2023-03-03T06:27:05.316 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-03-03T06:27:05.316 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42267sec preferred_lft 42267sec 2023-03-03T06:27:05.316 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2023-03-03T06:27:05.316 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:27:05.317 INFO:teuthology.orchestra.run.smithi003.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:27:05.317 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2023-03-03T06:27:05.317 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2023-03-03T06:27:05.317 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-03-03T06:27:05.317 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2023-03-03T06:27:05.317 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2023-03-03T06:27:05.317 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-03-03T06:27:05.317 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:69:76:c1:31 brd ff:ff:ff:ff:ff:ff 2023-03-03T06:27:05.318 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-03-03T06:27:05.318 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:27:05.318 INFO:teuthology.orchestra.run.smithi003.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-03T06:27:05.318 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:a2:d2:b0:b9:5c brd ff:ff:ff:ff:ff:ff 2023-03-03T06:27:05.318 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-03-03T06:27:05.318 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:27:05.318 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::f8ed:b6ff:fede:6b5f/64 scope link 2023-03-03T06:27:05.319 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:27:05.319 INFO:teuthology.orchestra.run.smithi003.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-03-03T06:27:05.319 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 8e:e5:8f:ab:8a:c0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-03-03T06:27:05.320 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::8ce5:8fff:feab:8ac0/64 scope link 2023-03-03T06:27:05.320 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:27:05.320 INFO:teuthology.orchestra.run.smithi003.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-03-03T06:27:05.320 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:a2:d2:b0:b9:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-03-03T06:27:05.320 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::a2:d2ff:feb0:b95c/64 scope link 2023-03-03T06:27:05.320 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2023-03-03T06:27:05.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:05.321 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-03-03T06:27:05.412 INFO:teuthology.orchestra.run.smithi003.stdout:1 2023-03-03T06:27:05.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:05.419 DEBUG:teuthology.orchestra.run.smithi003:> route 2023-03-03T06:27:05.486 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2023-03-03T06:27:05.486 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-03-03T06:27:05.486 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-03-03T06:27:05.486 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-03-03T06:27:05.487 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-03-03T06:27:05.487 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-03-03T06:27:05.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:05.488 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec bash -c ' 2023-03-03T06:27:05.488 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2023-03-03T06:27:05.489 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-03-03T06:27:05.489 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-03-03T06:27:05.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-03-03T06:27:05.490 DEBUG:teuthology.orchestra.run.smithi003:> ') 2023-03-03T06:27:05.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:05.649 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2023-03-03T06:27:05.671 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-03-03T06:27:05.671 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-03-03T06:27:05.673 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-03-03T06:27:05.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:05.674 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2023-03-03T06:27:05.759 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-03T06:27:05.759 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-03-03T06:27:05.824 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-03-03T06:27:05.825 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-03T06:27:05.826 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-03-03T06:27:05.890 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-03-03T06:27:05.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:05.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-03-03T06:27:06.052 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-03-03T06:27:06.053 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-03-03T06:27:06.053 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-03-03T06:27:06.094 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-03-03T06:27:06.094 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-03-03T06:27:06.094 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-03-03T06:27:06.094 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.32:3300,172.21.15.32:3301" 2023-03-03T06:27:06.095 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2023-03-03T06:27:06.120 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-03-03T06:27:06.121 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-03-03T06:27:06.121 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2023-03-03T06:27:06.121 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.32:3300,172.21.15.32:3301" 2023-03-03T06:27:06.121 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: switching to using v1 address with old syntax 2023-03-03T06:27:06.122 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.32:6789,172.21.15.32:6790" 2023-03-03T06:27:06.122 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:6789,172.21.15.32:6789,172.21.15.32:6790:/ 2023-03-03T06:27:06.122 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=7d496254-7268-405f-a1d8-c8033dc8659a" will pass to kernel 2023-03-03T06:27:06.136 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-03-03T06:27:06.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:06.137 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-03-03T06:27:06.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:06.183 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-03-03T06:27:06.284 INFO:teuthology.orchestra.run.smithi003.stdout:client55034 2023-03-03T06:27:06.284 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2023-03-03T06:27:06.285 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-03T06:27:06.346 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-03-03T06:27:06.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.598+0000 7f71a5066700 1 Processor -- start 2023-03-03T06:27:06.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.598+0000 7f71a5066700 1 -- start start 2023-03-03T06:27:06.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.602+0000 7f71a5066700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71a00cde60 0x7f71a00afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:06.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.602+0000 7f71a5066700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a015d8a0 con 0x7f71a00ce550 2023-03-03T06:27:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.602+0000 7f71a5066700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a00caa50 con 0x7f71a0159100 2023-03-03T06:27:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.602+0000 7f71a5066700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a00cb010 con 0x7f71a00cde60 2023-03-03T06:27:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.602+0000 7f71a4865700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f71a00ce550 0x7f71a00cc750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51724/0 (socket says 172.21.15.3:51724) 2023-03-03T06:27:06.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.602+0000 7f71a4865700 1 -- 172.21.15.3:0/2469998254 learned_addr learned my addr 172.21.15.3:0/2469998254 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:06.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.602+0000 7f71977fe700 1 --2- 172.21.15.3:0/2469998254 >> v2:172.21.15.3:3300/0 conn(0x7f71a00cde60 0x7f71a00afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:06.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.602+0000 7f719f7fe700 1 -- 172.21.15.3:0/2469998254 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 726723211 0 0) 0x7f71a015d8a0 con 0x7f71a00ce550 2023-03-03T06:27:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.602+0000 7f719f7fe700 1 -- 172.21.15.3:0/2469998254 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f718c0035d0 con 0x7f71a00ce550 2023-03-03T06:27:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.602+0000 7f71977fe700 1 -- 172.21.15.3:0/2469998254 >> v1:172.21.15.32:6789/0 conn(0x7f71a0159100 legacy=0x7f71a00ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.602+0000 7f71977fe700 1 -- 172.21.15.3:0/2469998254 >> v1:172.21.15.3:6789/0 conn(0x7f71a00ce550 legacy=0x7f71a00cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.602+0000 7f71977fe700 1 -- 172.21.15.3:0/2469998254 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a00cad30 con 0x7f71a00cde60 2023-03-03T06:27:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.606+0000 7f71977fe700 1 --2- 172.21.15.3:0/2469998254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a00cde60 0x7f71a00afb60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7188009850 tx=0x7f7188009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffcae602c4742dd server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:06.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.606+0000 7f719f7fe700 1 -- 172.21.15.3:0/2469998254 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188011830 con 0x7f71a00cde60 2023-03-03T06:27:06.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.606+0000 7f719f7fe700 1 -- 172.21.15.3:0/2469998254 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7188011990 con 0x7f71a00cde60 2023-03-03T06:27:06.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.606+0000 7f719f7fe700 1 -- 172.21.15.3:0/2469998254 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188011d40 con 0x7f71a00cde60 2023-03-03T06:27:06.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.606+0000 7f71a5066700 1 -- 172.21.15.3:0/2469998254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a00cde60 msgr2=0x7f71a00afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:06.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.606+0000 7f71a5066700 1 --2- 172.21.15.3:0/2469998254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a00cde60 0x7f71a00afb60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7188009850 tx=0x7f7188009b60 comp rx=0 tx=0).stop 2023-03-03T06:27:06.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.606+0000 7f71a5066700 1 -- 172.21.15.3:0/2469998254 shutdown_connections 2023-03-03T06:27:06.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.606+0000 7f71a5066700 1 --2- 172.21.15.3:0/2469998254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a00cde60 0x7f71a00afb60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:06.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.606+0000 7f71a5066700 1 -- 172.21.15.3:0/2469998254 >> 172.21.15.3:0/2469998254 conn(0x7f71a0070a30 msgr2=0x7f71a0070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:06.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.606+0000 7f71a5066700 1 -- 172.21.15.3:0/2469998254 shutdown_connections 2023-03-03T06:27:06.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.606+0000 7f71a5066700 1 -- 172.21.15.3:0/2469998254 wait complete. 2023-03-03T06:27:06.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.610+0000 7f71a5066700 1 Processor -- start 2023-03-03T06:27:06.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.610+0000 7f71a5066700 1 -- start start 2023-03-03T06:27:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.610+0000 7f71a5066700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71a00cde60 0x7f71a014cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.610+0000 7f71a5066700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a00ce550 0x7f71a00c4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:06.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.610+0000 7f71a5066700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a0159100 0x7f71a00b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:06.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.610+0000 7f71a5066700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00c49c0 con 0x7f71a00ce550 2023-03-03T06:27:06.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.610+0000 7f71a5066700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00b2580 con 0x7f71a0159100 2023-03-03T06:27:06.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.610+0000 7f71a5066700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a015a780 con 0x7f71a00cde60 2023-03-03T06:27:06.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f71977fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a00ce550 0x7f71a00c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:06.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f71977fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a00ce550 0x7f71a00c4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33766/0 (socket says 172.21.15.3:33766) 2023-03-03T06:27:06.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f71977fe700 1 -- 172.21.15.3:0/4249411225 learned_addr learned my addr 172.21.15.3:0/4249411225 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:06.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f71a4865700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a0159100 0x7f71a00b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:06.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f719ffff700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71a00cde60 0x7f71a014cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:06.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f71977fe700 1 -- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71a00cde60 msgr2=0x7f71a014cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:06.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f71977fe700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71a00cde60 0x7f71a014cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:06.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f71977fe700 1 -- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a0159100 msgr2=0x7f71a00b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:06.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f71977fe700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a0159100 0x7f71a00b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:06.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f71977fe700 1 -- 172.21.15.3:0/4249411225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71880094b0 con 0x7f71a00ce550 2023-03-03T06:27:06.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f71977fe700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a00ce550 0x7f71a00c4480 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7188000c00 tx=0x7f7188004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:06.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f719d7fa700 1 -- 172.21.15.3:0/4249411225 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718801a990 con 0x7f71a00ce550 2023-03-03T06:27:06.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f719d7fa700 1 -- 172.21.15.3:0/4249411225 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718801f070 con 0x7f71a00ce550 2023-03-03T06:27:06.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f719d7fa700 1 -- 172.21.15.3:0/4249411225 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718801ad30 con 0x7f71a00ce550 2023-03-03T06:27:06.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f71a5066700 1 -- 172.21.15.3:0/4249411225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a01594d0 con 0x7f71a00ce550 2023-03-03T06:27:06.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.614+0000 7f71a5066700 1 -- 172.21.15.3:0/4249411225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a015db60 con 0x7f71a00ce550 2023-03-03T06:27:06.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.618+0000 7f71a5066700 1 -- 172.21.15.3:0/4249411225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a0159660 con 0x7f71a00ce550 2023-03-03T06:27:06.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.618+0000 7f719d7fa700 1 -- 172.21.15.3:0/4249411225 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f718800f050 con 0x7f71a00ce550 2023-03-03T06:27:06.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.622+0000 7f719d7fa700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7180064ee0 0x7f7180067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:06.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.622+0000 7f719d7fa700 1 -- 172.21.15.3:0/4249411225 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7188083070 con 0x7f71a00ce550 2023-03-03T06:27:06.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.626+0000 7f719ffff700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7180064ee0 0x7f7180067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:06.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.626+0000 7f719ffff700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7180064ee0 0x7f7180067390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71900012e0 tx=0x7f7190009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:06.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.630+0000 7f719d7fa700 1 -- 172.21.15.3:0/4249411225 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f71880b3400 con 0x7f71a00ce550 2023-03-03T06:27:06.842 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.838+0000 7f71a5066700 1 -- 172.21.15.3:0/4249411225 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f71a0073e90 con 0x7f7180064ee0 2023-03-03T06:27:06.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.842+0000 7f719d7fa700 1 -- 172.21.15.3:0/4249411225 <== mgr.4105 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f71a0073e90 con 0x7f7180064ee0 2023-03-03T06:27:06.846 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2023-03-03T06:27:06.846 INFO:teuthology.orchestra.run.smithi003.stdout: { 2023-03-03T06:27:06.846 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephfs" 2023-03-03T06:27:06.847 INFO:teuthology.orchestra.run.smithi003.stdout: } 2023-03-03T06:27:06.847 INFO:teuthology.orchestra.run.smithi003.stdout:] 2023-03-03T06:27:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.846+0000 7f71a5066700 1 -- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7180064ee0 msgr2=0x7f7180067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:06.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.846+0000 7f71a5066700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7180064ee0 0x7f7180067390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71900012e0 tx=0x7f7190009040 comp rx=0 tx=0).stop 2023-03-03T06:27:06.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.846+0000 7f71a5066700 1 -- 172.21.15.3:0/4249411225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a00ce550 msgr2=0x7f71a00c4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:06.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.846+0000 7f71a5066700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a00ce550 0x7f71a00c4480 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7188000c00 tx=0x7f7188004770 comp rx=0 tx=0).stop 2023-03-03T06:27:06.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.846+0000 7f71a5066700 1 -- 172.21.15.3:0/4249411225 shutdown_connections 2023-03-03T06:27:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.846+0000 7f71a5066700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7180064ee0 0x7f7180067390 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.846+0000 7f71a5066700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a0159100 0x7f71a00b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.846+0000 7f71a5066700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71a00ce550 0x7f71a00c4480 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.846+0000 7f71a5066700 1 --2- 172.21.15.3:0/4249411225 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f71a00cde60 0x7f71a014cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.846+0000 7f71a5066700 1 -- 172.21.15.3:0/4249411225 >> 172.21.15.3:0/4249411225 conn(0x7f71a0070a30 msgr2=0x7f71a00cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:06.852 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.846+0000 7f71a5066700 1 -- 172.21.15.3:0/4249411225 shutdown_connections 2023-03-03T06:27:06.853 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:06.846+0000 7f71a5066700 1 -- 172.21.15.3:0/4249411225 wait complete. 2023-03-03T06:27:06.884 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-03-03T06:27:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.086+0000 7fc27a3f9700 1 Processor -- start 2023-03-03T06:27:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.090+0000 7fc27a3f9700 1 -- start start 2023-03-03T06:27:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.090+0000 7fc27a3f9700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc27406a780 0x7fc2740cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.090+0000 7fc27a3f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2740710c0 0x7fc2740ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.090+0000 7fc27a3f9700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27415c230 con 0x7fc2740cb9a0 2023-03-03T06:27:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.090+0000 7fc27a3f9700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2740caac0 con 0x7fc27406a780 2023-03-03T06:27:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.090+0000 7fc27a3f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740cd130 con 0x7fc2740710c0 2023-03-03T06:27:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.090+0000 7fc2793f7700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc27406a780 0x7fc2740cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.090+0000 7fc279bf8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2740710c0 0x7fc2740ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.090+0000 7fc278bf6700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc2740cb9a0 0x7fc2740709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:54756/0 (socket says 172.21.15.3:54756) 2023-03-03T06:27:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.090+0000 7fc278bf6700 1 -- 172.21.15.3:0/2633945763 learned_addr learned my addr 172.21.15.3:0/2633945763 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc279bf8700 1 -- 172.21.15.3:0/2633945763 >> v2:172.21.15.32:3300/0 conn(0x7fc27406a780 msgr2=0x7fc2740cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc279bf8700 1 --2- 172.21.15.3:0/2633945763 >> v2:172.21.15.32:3300/0 conn(0x7fc27406a780 0x7fc2740cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc279bf8700 1 -- 172.21.15.3:0/2633945763 >> v1:172.21.15.32:6789/0 conn(0x7fc2740cb9a0 legacy=0x7fc2740709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc279bf8700 1 -- 172.21.15.3:0/2633945763 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2740cbf60 con 0x7fc2740710c0 2023-03-03T06:27:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc279bf8700 1 --2- 172.21.15.3:0/2633945763 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2740710c0 0x7fc2740ce5e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009850 tx=0x7fc26c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70a6ea997a4e18ab server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc26bfff700 1 -- 172.21.15.3:0/2633945763 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c011830 con 0x7fc2740710c0 2023-03-03T06:27:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc26bfff700 1 -- 172.21.15.3:0/2633945763 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26c011990 con 0x7fc2740710c0 2023-03-03T06:27:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc26bfff700 1 -- 172.21.15.3:0/2633945763 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c011d40 con 0x7fc2740710c0 2023-03-03T06:27:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc27a3f9700 1 -- 172.21.15.3:0/2633945763 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2740710c0 msgr2=0x7fc2740ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc27a3f9700 1 --2- 172.21.15.3:0/2633945763 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2740710c0 0x7fc2740ce5e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009850 tx=0x7fc26c009b60 comp rx=0 tx=0).stop 2023-03-03T06:27:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc27a3f9700 1 -- 172.21.15.3:0/2633945763 shutdown_connections 2023-03-03T06:27:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc27a3f9700 1 --2- 172.21.15.3:0/2633945763 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2740710c0 0x7fc2740ce5e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc27a3f9700 1 --2- 172.21.15.3:0/2633945763 >> v2:172.21.15.32:3300/0 conn(0x7fc27406a780 0x7fc2740cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.094+0000 7fc27a3f9700 1 -- 172.21.15.3:0/2633945763 >> 172.21.15.3:0/2633945763 conn(0x7fc2740607c0 msgr2=0x7fc274060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.098+0000 7fc27a3f9700 1 -- 172.21.15.3:0/2633945763 shutdown_connections 2023-03-03T06:27:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.098+0000 7fc27a3f9700 1 -- 172.21.15.3:0/2633945763 wait complete. 2023-03-03T06:27:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc27a3f9700 1 Processor -- start 2023-03-03T06:27:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc27a3f9700 1 -- start start 2023-03-03T06:27:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc27a3f9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27406a780 0x7fc274152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc27a3f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2740710c0 0x7fc27415bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc27a3f9700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2740cb9a0 0x7fc2740c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc27a3f9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc274151850 con 0x7fc27406a780 2023-03-03T06:27:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc27a3f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27415c000 con 0x7fc2740710c0 2023-03-03T06:27:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc27a3f9700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740c9930 con 0x7fc2740cb9a0 2023-03-03T06:27:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc2793f7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27406a780 0x7fc274152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:07.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc2793f7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27406a780 0x7fc274152580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33776/0 (socket says 172.21.15.3:33776) 2023-03-03T06:27:07.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc2793f7700 1 -- 172.21.15.3:0/1254557989 learned_addr learned my addr 172.21.15.3:0/1254557989 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:07.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc278bf6700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2740710c0 0x7fc27415bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:07.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc279bf8700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2740cb9a0 0x7fc2740c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:07.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc2793f7700 1 -- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2740cb9a0 msgr2=0x7fc2740c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:07.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc2793f7700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2740cb9a0 0x7fc2740c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:07.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc2793f7700 1 -- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2740710c0 msgr2=0x7fc27415bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:07.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc2793f7700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2740710c0 0x7fc27415bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:07.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc2793f7700 1 -- 172.21.15.3:0/1254557989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c0094b0 con 0x7fc27406a780 2023-03-03T06:27:07.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.102+0000 7fc278bf6700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2740710c0 0x7fc27415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:27:07.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.106+0000 7fc2793f7700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27406a780 0x7fc274152580 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc270009db0 tx=0x7fc27000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.106+0000 7fc269ffb700 1 -- 172.21.15.3:0/1254557989 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270009c20 con 0x7fc27406a780 2023-03-03T06:27:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.106+0000 7fc269ffb700 1 -- 172.21.15.3:0/1254557989 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27000f040 con 0x7fc27406a780 2023-03-03T06:27:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.106+0000 7fc27a3f9700 1 -- 172.21.15.3:0/1254557989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27415c7d0 con 0x7fc27406a780 2023-03-03T06:27:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.106+0000 7fc269ffb700 1 -- 172.21.15.3:0/1254557989 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270014480 con 0x7fc27406a780 2023-03-03T06:27:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.106+0000 7fc27a3f9700 1 -- 172.21.15.3:0/1254557989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27415c4f0 con 0x7fc27406a780 2023-03-03T06:27:07.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.106+0000 7fc27a3f9700 1 -- 172.21.15.3:0/1254557989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27415cab0 con 0x7fc27406a780 2023-03-03T06:27:07.114 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.110+0000 7fc269ffb700 1 -- 172.21.15.3:0/1254557989 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc2700145e0 con 0x7fc27406a780 2023-03-03T06:27:07.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.110+0000 7fc269ffb700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc258064ee0 0x7fc258067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:07.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.110+0000 7fc269ffb700 1 -- 172.21.15.3:0/1254557989 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc270082df0 con 0x7fc27406a780 2023-03-03T06:27:07.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.114+0000 7fc278bf6700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc258064ee0 0x7fc258067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:07.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.114+0000 7fc278bf6700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc258064ee0 0x7fc258067390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc264007fd0 tx=0x7fc264009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:07.123 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.118+0000 7fc269ffb700 1 -- 172.21.15.3:0/1254557989 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc270050d60 con 0x7fc27406a780 2023-03-03T06:27:07.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.334+0000 7fc27a3f9700 1 -- 172.21.15.3:0/1254557989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc274073e90 con 0x7fc27406a780 2023-03-03T06:27:07.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.722+0000 7fc269ffb700 1 -- 172.21.15.3:0/1254557989 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 6) v1 ==== 83976+0+0 (secure 0 0 0) 0x7fc270051130 con 0x7fc27406a780 2023-03-03T06:27:07.727 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.722+0000 7fc269ffb700 1 -- 172.21.15.3:0/1254557989 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc270050d60 con 0x7fc27406a780 2023-03-03T06:27:07.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.726+0000 7fc27a3f9700 1 -- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc258064ee0 msgr2=0x7fc258067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:07.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.726+0000 7fc27a3f9700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc258064ee0 0x7fc258067390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc264007fd0 tx=0x7fc264009040 comp rx=0 tx=0).stop 2023-03-03T06:27:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.726+0000 7fc27a3f9700 1 -- 172.21.15.3:0/1254557989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27406a780 msgr2=0x7fc274152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.726+0000 7fc27a3f9700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27406a780 0x7fc274152580 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc270009db0 tx=0x7fc27000c3a0 comp rx=0 tx=0).stop 2023-03-03T06:27:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.726+0000 7fc27a3f9700 1 -- 172.21.15.3:0/1254557989 shutdown_connections 2023-03-03T06:27:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.726+0000 7fc27a3f9700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc258064ee0 0x7fc258067390 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.726+0000 7fc27a3f9700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc2740cb9a0 0x7fc2740c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.726+0000 7fc27a3f9700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2740710c0 0x7fc27415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:07.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.726+0000 7fc27a3f9700 1 --2- 172.21.15.3:0/1254557989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27406a780 0x7fc274152580 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:07.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.726+0000 7fc27a3f9700 1 -- 172.21.15.3:0/1254557989 >> 172.21.15.3:0/1254557989 conn(0x7fc2740607c0 msgr2=0x7fc2740ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:07.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.726+0000 7fc27a3f9700 1 -- 172.21.15.3:0/1254557989 shutdown_connections 2023-03-03T06:27:07.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.726+0000 7fc27a3f9700 1 -- 172.21.15.3:0/1254557989 wait complete. 2023-03-03T06:27:07.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-03-03T06:27:07.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.990+0000 7f4c655d2700 1 Processor -- start 2023-03-03T06:27:07.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.990+0000 7f4c655d2700 1 -- start start 2023-03-03T06:27:08.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.994+0000 7f4c655d2700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4c600b6050 0x7f4c600afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:08.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.994+0000 7f4c655d2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c600d12b0 con 0x7f4c600caf20 2023-03-03T06:27:08.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.994+0000 7f4c577fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4c600caf20 0x7f4c6015dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51752/0 (socket says 172.21.15.3:51752) 2023-03-03T06:27:08.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.994+0000 7f4c577fe700 1 -- 172.21.15.3:0/3333864022 learned_addr learned my addr 172.21.15.3:0/3333864022 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:08.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.994+0000 7f4c655d2700 1 -- 172.21.15.3:0/3333864022 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c600cb6e0 con 0x7f4c6015e480 2023-03-03T06:27:08.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.994+0000 7f4c655d2700 1 -- 172.21.15.3:0/3333864022 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c600d0c80 con 0x7f4c600b6050 2023-03-03T06:27:08.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.994+0000 7f4c5ffff700 1 --2- 172.21.15.3:0/3333864022 >> v2:172.21.15.3:3300/0 conn(0x7f4c600b6050 0x7f4c600afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:08.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c5f7fe700 1 -- 172.21.15.3:0/3333864022 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485714479 0 0) 0x7f4c600d12b0 con 0x7f4c600caf20 2023-03-03T06:27:08.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c5f7fe700 1 -- 172.21.15.3:0/3333864022 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c4c0035d0 con 0x7f4c600caf20 2023-03-03T06:27:08.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c5ffff700 1 -- 172.21.15.3:0/3333864022 >> v1:172.21.15.32:6789/0 conn(0x7f4c6015e480 legacy=0x7f4c600ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:08.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c5ffff700 1 -- 172.21.15.3:0/3333864022 >> v1:172.21.15.3:6789/0 conn(0x7f4c600caf20 legacy=0x7f4c6015dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:08.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c5ffff700 1 -- 172.21.15.3:0/3333864022 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c600b6420 con 0x7f4c600b6050 2023-03-03T06:27:08.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c5ffff700 1 --2- 172.21.15.3:0/3333864022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c600b6050 0x7f4c600afda0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009850 tx=0x7f4c50009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db177f545af5b84 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:08.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c5f7fe700 1 -- 172.21.15.3:0/3333864022 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50011830 con 0x7f4c600b6050 2023-03-03T06:27:08.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c5f7fe700 1 -- 172.21.15.3:0/3333864022 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c50011990 con 0x7f4c600b6050 2023-03-03T06:27:08.004 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c5f7fe700 1 -- 172.21.15.3:0/3333864022 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50011d40 con 0x7f4c600b6050 2023-03-03T06:27:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c655d2700 1 -- 172.21.15.3:0/3333864022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c600b6050 msgr2=0x7f4c600afda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c655d2700 1 --2- 172.21.15.3:0/3333864022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c600b6050 0x7f4c600afda0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009850 tx=0x7f4c50009b60 comp rx=0 tx=0).stop 2023-03-03T06:27:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c655d2700 1 -- 172.21.15.3:0/3333864022 shutdown_connections 2023-03-03T06:27:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c655d2700 1 --2- 172.21.15.3:0/3333864022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c600b6050 0x7f4c600afda0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c655d2700 1 -- 172.21.15.3:0/3333864022 >> 172.21.15.3:0/3333864022 conn(0x7f4c6006e5c0 msgr2=0x7f4c6006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c655d2700 1 -- 172.21.15.3:0/3333864022 shutdown_connections 2023-03-03T06:27:08.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:07.998+0000 7f4c655d2700 1 -- 172.21.15.3:0/3333864022 wait complete. 2023-03-03T06:27:08.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.002+0000 7f4c655d2700 1 Processor -- start 2023-03-03T06:27:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.002+0000 7f4c655d2700 1 -- start start 2023-03-03T06:27:08.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c655d2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c600b6050 0x7f4c60157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:08.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c655d2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c600caf20 0x7f4c60156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:08.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c655d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c6015e480 0x7f4c600c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:08.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c655d2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600c7170 con 0x7f4c600caf20 2023-03-03T06:27:08.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c655d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c60157250 con 0x7f4c6015e480 2023-03-03T06:27:08.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c655d2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6015a740 con 0x7f4c600b6050 2023-03-03T06:27:08.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c577fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c600caf20 0x7f4c60156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:08.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c5ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c600b6050 0x7f4c60157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:08.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c64dd1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c6015e480 0x7f4c600c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:08.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c5ffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c600b6050 0x7f4c60157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:36730/0 (socket says 172.21.15.3:36730) 2023-03-03T06:27:08.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c5ffff700 1 -- 172.21.15.3:0/3120072053 learned_addr learned my addr 172.21.15.3:0/3120072053 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:08.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c5ffff700 1 -- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c6015e480 msgr2=0x7f4c600c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:08.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c5ffff700 1 --2- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c6015e480 0x7f4c600c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c5ffff700 1 -- 172.21.15.3:0/3120072053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c600caf20 msgr2=0x7f4c60156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:08.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c5ffff700 1 --2- 172.21.15.3:0/3120072053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c600caf20 0x7f4c60156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c5ffff700 1 -- 172.21.15.3:0/3120072053 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c500094b0 con 0x7f4c600b6050 2023-03-03T06:27:08.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c5ffff700 1 --2- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c600b6050 0x7f4c60157460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4c500090b0 tx=0x7f4c50004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:08.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c5d7fa700 1 -- 172.21.15.3:0/3120072053 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5001a990 con 0x7f4c600b6050 2023-03-03T06:27:08.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c5d7fa700 1 -- 172.21.15.3:0/3120072053 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5001f070 con 0x7f4c600b6050 2023-03-03T06:27:08.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c5d7fa700 1 -- 172.21.15.3:0/3120072053 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5001ad30 con 0x7f4c600b6050 2023-03-03T06:27:08.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c655d2700 1 -- 172.21.15.3:0/3120072053 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c600d0f40 con 0x7f4c600b6050 2023-03-03T06:27:08.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.006+0000 7f4c655d2700 1 -- 172.21.15.3:0/3120072053 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c600cb400 con 0x7f4c600b6050 2023-03-03T06:27:08.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.010+0000 7f4c655d2700 1 -- 172.21.15.3:0/3120072053 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c60073e90 con 0x7f4c600b6050 2023-03-03T06:27:08.019 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.014+0000 7f4c5d7fa700 1 -- 172.21.15.3:0/3120072053 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 6) v1 ==== 83976+0+0 (secure 0 0 0) 0x7f4c5000f050 con 0x7f4c600b6050 2023-03-03T06:27:08.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.014+0000 7f4c5d7fa700 1 --2- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c40064f30 0x7f4c400673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:08.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.014+0000 7f4c5d7fa700 1 -- 172.21.15.3:0/3120072053 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4c50083190 con 0x7f4c600b6050 2023-03-03T06:27:08.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.014+0000 7f4c577fe700 1 -- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c40064f30 msgr2=0x7f4c400673e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/32113 2023-03-03T06:27:08.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.014+0000 7f4c577fe700 1 --2- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c40064f30 0x7f4c400673e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:27:08.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.026+0000 7f4c5d7fa700 1 -- 172.21.15.3:0/3120072053 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4c50051100 con 0x7f4c600b6050 2023-03-03T06:27:08.038 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:08.031+0000 7f44dcc482c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-03T06:27:08.044 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:08.038+0000 7fabcc9982c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-03T06:27:08.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.210+0000 7f4c655d2700 1 -- 172.21.15.3:0/3120072053 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f4c600b2e00 con 0x7f4c600b6050 2023-03-03T06:27:08.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.218+0000 7f4c577fe700 1 -- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c40064f30 msgr2=0x7f4c400673e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/32113 2023-03-03T06:27:08.222 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.218+0000 7f4c577fe700 1 --2- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c40064f30 0x7f4c400673e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-03-03T06:27:08.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.222+0000 7f4c5d7fa700 1 -- 172.21.15.3:0/3120072053 <== mon.2 v2:172.21.15.32:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4c500210d0 con 0x7f4c600b6050 2023-03-03T06:27:08.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.226+0000 7f4c655d2700 1 -- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c40064f30 msgr2=0x7f4c400673e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:27:08.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.226+0000 7f4c655d2700 1 --2- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c40064f30 0x7f4c400673e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.226+0000 7f4c655d2700 1 -- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c600b6050 msgr2=0x7f4c60157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:08.232 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.226+0000 7f4c655d2700 1 --2- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c600b6050 0x7f4c60157460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4c500090b0 tx=0x7f4c50004770 comp rx=0 tx=0).stop 2023-03-03T06:27:08.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.226+0000 7f4c655d2700 1 -- 172.21.15.3:0/3120072053 shutdown_connections 2023-03-03T06:27:08.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.226+0000 7f4c655d2700 1 --2- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c40064f30 0x7f4c400673e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.226+0000 7f4c655d2700 1 --2- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c6015e480 0x7f4c600c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.226+0000 7f4c655d2700 1 --2- 172.21.15.3:0/3120072053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c600caf20 0x7f4c60156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.226+0000 7f4c655d2700 1 --2- 172.21.15.3:0/3120072053 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4c600b6050 0x7f4c60157460 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.226+0000 7f4c655d2700 1 -- 172.21.15.3:0/3120072053 >> 172.21.15.3:0/3120072053 conn(0x7f4c6006e5c0 msgr2=0x7f4c600ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:08.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.230+0000 7f4c655d2700 1 -- 172.21.15.3:0/3120072053 shutdown_connections 2023-03-03T06:27:08.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.230+0000 7f4c655d2700 1 -- 172.21.15.3:0/3120072053 wait complete. 2023-03-03T06:27:08.266 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-03-03T06:27:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.470+0000 7fdad93a5700 1 Processor -- start 2023-03-03T06:27:08.475 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.470+0000 7fdad93a5700 1 -- start start 2023-03-03T06:27:08.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.470+0000 7fdad93a5700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdad40b6050 0x7fdad40d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:08.477 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.470+0000 7fdad93a5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad415c960 0x7fdad40b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:08.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.470+0000 7fdad93a5700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad40d32c0 con 0x7fdad40d4040 2023-03-03T06:27:08.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.470+0000 7fdad93a5700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad40b6960 con 0x7fdad40b6050 2023-03-03T06:27:08.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.470+0000 7fdad93a5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40d26b0 con 0x7fdad415c960 2023-03-03T06:27:08.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.470+0000 7fdad2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdad40b6050 0x7fdad40d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:08.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdad40b6050 0x7fdad40d3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33812/0 (socket says 172.21.15.3:33812) 2023-03-03T06:27:08.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad2ffd700 1 -- 172.21.15.3:0/897302942 learned_addr learned my addr 172.21.15.3:0/897302942 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:08.478 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdacbfff700 1 --2- 172.21.15.3:0/897302942 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad415c960 0x7fdad40b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:08.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad2ffd700 1 -- 172.21.15.3:0/897302942 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad415c960 msgr2=0x7fdad40b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:08.479 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad2ffd700 1 --2- 172.21.15.3:0/897302942 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad415c960 0x7fdad40b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad2ffd700 1 -- 172.21.15.3:0/897302942 >> v1:172.21.15.3:6789/0 conn(0x7fdad40d4040 legacy=0x7fdad406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:08.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad2ffd700 1 -- 172.21.15.3:0/897302942 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad40d2990 con 0x7fdad40b6050 2023-03-03T06:27:08.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad2ffd700 1 --2- 172.21.15.3:0/897302942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad40b6050 0x7fdad40d3960 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009e80 tx=0x7fdac400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3978a161d7748a94 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:08.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad27fc700 1 -- 172.21.15.3:0/897302942 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac400e070 con 0x7fdad40b6050 2023-03-03T06:27:08.480 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad27fc700 1 -- 172.21.15.3:0/897302942 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac400b980 con 0x7fdad40b6050 2023-03-03T06:27:08.481 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad27fc700 1 -- 172.21.15.3:0/897302942 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac400bd30 con 0x7fdad40b6050 2023-03-03T06:27:08.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad93a5700 1 -- 172.21.15.3:0/897302942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad40b6050 msgr2=0x7fdad40d3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:08.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad93a5700 1 --2- 172.21.15.3:0/897302942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad40b6050 0x7fdad40d3960 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009e80 tx=0x7fdac400b330 comp rx=0 tx=0).stop 2023-03-03T06:27:08.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad93a5700 1 -- 172.21.15.3:0/897302942 shutdown_connections 2023-03-03T06:27:08.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad93a5700 1 --2- 172.21.15.3:0/897302942 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad415c960 0x7fdad40b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad93a5700 1 --2- 172.21.15.3:0/897302942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad40b6050 0x7fdad40d3960 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad93a5700 1 -- 172.21.15.3:0/897302942 >> 172.21.15.3:0/897302942 conn(0x7fdad406a9b0 msgr2=0x7fdad40d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:08.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad93a5700 1 -- 172.21.15.3:0/897302942 shutdown_connections 2023-03-03T06:27:08.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.474+0000 7fdad93a5700 1 -- 172.21.15.3:0/897302942 wait complete. 2023-03-03T06:27:08.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.478+0000 7fdad93a5700 1 Processor -- start 2023-03-03T06:27:08.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.478+0000 7fdad93a5700 1 -- start start 2023-03-03T06:27:08.486 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdad93a5700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad40b6050 0x7fdad415de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdad93a5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad40d4040 0x7fdad4147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdad93a5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad415c960 0x7fdad40d45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdad93a5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad4155700 con 0x7fdad40d4040 2023-03-03T06:27:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdad93a5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad415f220 con 0x7fdad415c960 2023-03-03T06:27:08.487 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdad93a5700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad415e6a0 con 0x7fdad40b6050 2023-03-03T06:27:08.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdacbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad40d4040 0x7fdad4147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:08.488 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdacbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad40d4040 0x7fdad4147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33816/0 (socket says 172.21.15.3:33816) 2023-03-03T06:27:08.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdacbfff700 1 -- 172.21.15.3:0/3808759874 learned_addr learned my addr 172.21.15.3:0/3808759874 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:08.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdad2ffd700 1 --2- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad40b6050 0x7fdad415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:08.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdad3fff700 1 --2- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad415c960 0x7fdad40d45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:08.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdacbfff700 1 -- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad40b6050 msgr2=0x7fdad415de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:08.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdacbfff700 1 --2- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad40b6050 0x7fdad415de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.489 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdacbfff700 1 -- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad415c960 msgr2=0x7fdad40d45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:08.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdacbfff700 1 --2- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad415c960 0x7fdad40d45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdacbfff700 1 -- 172.21.15.3:0/3808759874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac4009b30 con 0x7fdad40d4040 2023-03-03T06:27:08.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdacbfff700 1 --2- 172.21.15.3:0/3808759874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad40d4040 0x7fdad4147790 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00d7d0 tx=0x7fdabc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:08.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdacb7fe700 1 -- 172.21.15.3:0/3808759874 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc017070 con 0x7fdad40d4040 2023-03-03T06:27:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdacb7fe700 1 -- 172.21.15.3:0/3808759874 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdabc004020 con 0x7fdad40d4040 2023-03-03T06:27:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdad93a5700 1 -- 172.21.15.3:0/3808759874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad40d2f50 con 0x7fdad40d4040 2023-03-03T06:27:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdacb7fe700 1 -- 172.21.15.3:0/3808759874 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0041b0 con 0x7fdad40d4040 2023-03-03T06:27:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.482+0000 7fdad93a5700 1 -- 172.21.15.3:0/3808759874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad40d2c70 con 0x7fdad40d4040 2023-03-03T06:27:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.486+0000 7fdad93a5700 1 -- 172.21.15.3:0/3808759874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad40cf7f0 con 0x7fdad40d4040 2023-03-03T06:27:08.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.486+0000 7fdacb7fe700 1 -- 172.21.15.3:0/3808759874 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 6) v1 ==== 83976+0+0 (secure 0 0 0) 0x7fdabc004310 con 0x7fdad40d4040 2023-03-03T06:27:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.490+0000 7fdacb7fe700 1 --2- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdab4064f30 0x7fdab40673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.490+0000 7fdacb7fe700 1 -- 172.21.15.3:0/3808759874 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fdabc081c70 con 0x7fdad40d4040 2023-03-03T06:27:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.490+0000 7fdad2ffd700 1 -- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdab4064f30 msgr2=0x7fdab40673e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/32113 2023-03-03T06:27:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.490+0000 7fdad2ffd700 1 --2- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdab4064f30 0x7fdab40673e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:27:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.494+0000 7fdacb7fe700 1 -- 172.21.15.3:0/3808759874 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fdabc04fbe0 con 0x7fdad40d4040 2023-03-03T06:27:08.658 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:08.651+0000 7f44dcc482c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-03T06:27:08.677 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:08.670+0000 7fabcc9982c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-03T06:27:08.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.674+0000 7fdad93a5700 1 -- 172.21.15.3:0/3808759874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fdad404e180 con 0x7fdad40d4040 2023-03-03T06:27:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.682+0000 7fdacb7fe700 1 -- 172.21.15.3:0/3808759874 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7fdabc053200 con 0x7fdad40d4040 2023-03-03T06:27:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.686+0000 7fdad93a5700 1 -- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdab4064f30 msgr2=0x7fdab40673e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:27:08.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.686+0000 7fdad93a5700 1 --2- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdab4064f30 0x7fdab40673e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.686+0000 7fdad93a5700 1 -- 172.21.15.3:0/3808759874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad40d4040 msgr2=0x7fdad4147790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:08.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.686+0000 7fdad93a5700 1 --2- 172.21.15.3:0/3808759874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad40d4040 0x7fdad4147790 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00d7d0 tx=0x7fdabc00dae0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.686+0000 7fdad93a5700 1 -- 172.21.15.3:0/3808759874 shutdown_connections 2023-03-03T06:27:08.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.686+0000 7fdad93a5700 1 --2- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fdab4064f30 0x7fdab40673e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.686+0000 7fdad93a5700 1 --2- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdad415c960 0x7fdad40d45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.686+0000 7fdad93a5700 1 --2- 172.21.15.3:0/3808759874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad40d4040 0x7fdad4147790 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.686+0000 7fdad93a5700 1 --2- 172.21.15.3:0/3808759874 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fdad40b6050 0x7fdad415de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:08.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.686+0000 7fdad93a5700 1 -- 172.21.15.3:0/3808759874 >> 172.21.15.3:0/3808759874 conn(0x7fdad406a9b0 msgr2=0x7fdad415a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:08.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.686+0000 7fdad93a5700 1 -- 172.21.15.3:0/3808759874 shutdown_connections 2023-03-03T06:27:08.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:08.686+0000 7fdad93a5700 1 -- 172.21.15.3:0/3808759874 wait complete. 2023-03-03T06:27:08.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:08.724 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-03-03T06:27:08.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:08.755 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-03-03T06:27:08.757 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:08.751+0000 7f44dcc482c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-03T06:27:08.777 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:08.770+0000 7fabcc9982c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-03T06:27:08.810 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir0 1M 2023-03-03T06:27:08.931 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:08.923+0000 7f44dcc482c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-03T06:27:08.953 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:08.946+0000 7fabcc9982c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-03T06:27:09.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.054+0000 7f8ff5534700 1 Processor -- start 2023-03-03T06:27:09.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.054+0000 7f8ff5534700 1 -- start start 2023-03-03T06:27:09.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8ff5534700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ff00ccc70 0x7f8ff00c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:09.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8ff5534700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff006c090 con 0x7f8ff00c9520 2023-03-03T06:27:09.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8ff5534700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff00cd230 con 0x7f8ff0158810 2023-03-03T06:27:09.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8ff5534700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff006c2e0 con 0x7f8ff00ccc70 2023-03-03T06:27:09.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8fef7fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ff00ccc70 0x7f8ff00c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:09.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8feffff700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8ff0158810 0x7f8ff015ce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:54800/0 (socket says 172.21.15.3:54800) 2023-03-03T06:27:09.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8feffff700 1 -- 172.21.15.3:0/2441916096 learned_addr learned my addr 172.21.15.3:0/2441916096 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:09.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8ff4d33700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8ff00c9520 0x7f8ff00709b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51766/0 (socket says 172.21.15.3:51766) 2023-03-03T06:27:09.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8fef7fe700 1 -- 172.21.15.3:0/2441916096 >> v1:172.21.15.32:6789/0 conn(0x7f8ff0158810 legacy=0x7f8ff015ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:09.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8fef7fe700 1 -- 172.21.15.3:0/2441916096 >> v1:172.21.15.3:6789/0 conn(0x7f8ff00c9520 legacy=0x7f8ff00709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:09.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8fef7fe700 1 -- 172.21.15.3:0/2441916096 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff006c5c0 con 0x7f8ff00ccc70 2023-03-03T06:27:09.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8fef7fe700 1 --2- 172.21.15.3:0/2441916096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ccc70 0x7f8ff00c8fe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009850 tx=0x7f8fe4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41a5b6d0c6b0108 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:09.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8feeffd700 1 -- 172.21.15.3:0/2441916096 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400f940 con 0x7f8ff00ccc70 2023-03-03T06:27:09.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8feeffd700 1 -- 172.21.15.3:0/2441916096 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe400faa0 con 0x7f8ff00ccc70 2023-03-03T06:27:09.065 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.058+0000 7f8feeffd700 1 -- 172.21.15.3:0/2441916096 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400fe30 con 0x7f8ff00ccc70 2023-03-03T06:27:09.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.062+0000 7f8ff5534700 1 -- 172.21.15.3:0/2441916096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ccc70 msgr2=0x7f8ff00c8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:09.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.062+0000 7f8ff5534700 1 --2- 172.21.15.3:0/2441916096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ccc70 0x7f8ff00c8fe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009850 tx=0x7f8fe4009b60 comp rx=0 tx=0).stop 2023-03-03T06:27:09.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.062+0000 7f8ff5534700 1 -- 172.21.15.3:0/2441916096 shutdown_connections 2023-03-03T06:27:09.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.062+0000 7f8ff5534700 1 --2- 172.21.15.3:0/2441916096 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ccc70 0x7f8ff00c8fe0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:09.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.062+0000 7f8ff5534700 1 -- 172.21.15.3:0/2441916096 >> 172.21.15.3:0/2441916096 conn(0x7f8ff00607c0 msgr2=0x7f8ff0060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:09.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.062+0000 7f8ff5534700 1 -- 172.21.15.3:0/2441916096 shutdown_connections 2023-03-03T06:27:09.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.062+0000 7f8ff5534700 1 -- 172.21.15.3:0/2441916096 wait complete. 2023-03-03T06:27:09.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.062+0000 7f8ff5534700 1 Processor -- start 2023-03-03T06:27:09.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.062+0000 7f8ff5534700 1 -- start start 2023-03-03T06:27:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8ff5534700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00c9520 0x7f8ff00c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8ff5534700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8ff00ccc70 0x7f8ff00c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8ff5534700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff0158810 0x7f8ff00aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8ff5534700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00b00e0 con 0x7f8ff00c9520 2023-03-03T06:27:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8ff5534700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff0158e10 con 0x7f8ff0158810 2023-03-03T06:27:09.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8ff5534700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01590f0 con 0x7f8ff00ccc70 2023-03-03T06:27:09.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8feffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00c9520 0x7f8ff00c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:09.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8feffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00c9520 0x7f8ff00c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33840/0 (socket says 172.21.15.3:33840) 2023-03-03T06:27:09.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8feffff700 1 -- 172.21.15.3:0/1960315799 learned_addr learned my addr 172.21.15.3:0/1960315799 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:09.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8ff4d33700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff0158810 0x7f8ff00aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:09.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8fef7fe700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8ff00ccc70 0x7f8ff00c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:09.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8feffff700 1 -- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8ff00ccc70 msgr2=0x7f8ff00c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:09.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8feffff700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8ff00ccc70 0x7f8ff00c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:09.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8feffff700 1 -- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff0158810 msgr2=0x7f8ff00aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8feffff700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff0158810 0x7f8ff00aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8feffff700 1 -- 172.21.15.3:0/1960315799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe40094b0 con 0x7f8ff00c9520 2023-03-03T06:27:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8feffff700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00c9520 0x7f8ff00c8350 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0001eb0 tx=0x7f8fe000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:09.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8fecff9700 1 -- 172.21.15.3:0/1960315799 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0005d90 con 0x7f8ff00c9520 2023-03-03T06:27:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8fecff9700 1 -- 172.21.15.3:0/1960315799 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe00070d0 con 0x7f8ff00c9520 2023-03-03T06:27:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8fecff9700 1 -- 172.21.15.3:0/1960315799 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0007440 con 0x7f8ff00c9520 2023-03-03T06:27:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8ff5534700 1 -- 172.21.15.3:0/1960315799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff0158be0 con 0x7f8ff00c9520 2023-03-03T06:27:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.066+0000 7f8ff5534700 1 -- 172.21.15.3:0/1960315799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff006c8a0 con 0x7f8ff00c9520 2023-03-03T06:27:09.074 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.070+0000 7f8ff5534700 1 -- 172.21.15.3:0/1960315799 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff00710c0 con 0x7f8ff00c9520 2023-03-03T06:27:09.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.070+0000 7f8fecff9700 1 -- 172.21.15.3:0/1960315799 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 6) v1 ==== 83976+0+0 (secure 0 0 0) 0x7f8fe0019030 con 0x7f8ff00c9520 2023-03-03T06:27:09.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.074+0000 7f8fecff9700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 0x7f8fdc0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:09.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.074+0000 7f8fecff9700 1 -- 172.21.15.3:0/1960315799 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8fe0081390 con 0x7f8ff00c9520 2023-03-03T06:27:09.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.074+0000 7f8fef7fe700 1 -- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 msgr2=0x7f8fdc0673e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/32113 2023-03-03T06:27:09.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.074+0000 7f8fef7fe700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 0x7f8fdc0673e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-03T06:27:09.084 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.078+0000 7f8fecff9700 1 -- 172.21.15.3:0/1960315799 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f8fe004a400 con 0x7f8ff00c9520 2023-03-03T06:27:09.262 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:09.255+0000 7f44dcc482c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-03T06:27:09.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.278+0000 7f8fef7fe700 1 -- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 msgr2=0x7f8fdc0673e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/32113 2023-03-03T06:27:09.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.278+0000 7f8fef7fe700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 0x7f8fdc0673e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-03-03T06:27:09.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.278+0000 7f8ff5534700 1 -- 172.21.15.3:0/1960315799 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ff015b780 con 0x7f8fdc064f30 2023-03-03T06:27:09.290 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:09.286+0000 7fabcc9982c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-03T06:27:09.364 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:09.359+0000 7f44dcc482c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-03T06:27:09.391 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:09.386+0000 7fabcc9982c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-03T06:27:09.637 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:09.631+0000 7f44dcc482c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-03T06:27:09.666 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:09.662+0000 7fabcc9982c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-03T06:27:09.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.682+0000 7f8fef7fe700 1 -- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 msgr2=0x7f8fdc0673e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/32113 2023-03-03T06:27:09.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:09.682+0000 7f8fef7fe700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 0x7f8fdc0673e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-03-03T06:27:09.750 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:09.743+0000 7f44dcc482c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-03T06:27:09.781 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:09.774+0000 7fabcc9982c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-03T06:27:09.864 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:09.859+0000 7f44dcc482c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-03T06:27:09.895 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:09.890+0000 7fabcc9982c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-03T06:27:10.337 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:10.331+0000 7f44dcc482c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-03T06:27:10.372 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:10.366+0000 7fabcc9982c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-03T06:27:10.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:10.486+0000 7f8fef7fe700 1 -- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 msgr2=0x7f8fdc0673e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/32113 2023-03-03T06:27:10.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:10.486+0000 7f8fef7fe700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 0x7f8fdc0673e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-03-03T06:27:10.562 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:10.555+0000 7f44dcc482c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-03T06:27:10.591 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:10.586+0000 7fabcc9982c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-03T06:27:11.245 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:11.238+0000 7fabcc9982c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-03T06:27:11.257 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:11.251+0000 7f44dcc482c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-03T06:27:11.531 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:11.526+0000 7fabcc9982c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-03T06:27:11.542 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:11.535+0000 7f44dcc482c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-03T06:27:11.780 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:11.774+0000 7fabcc9982c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-03T06:27:11.793 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:11.787+0000 7f44dcc482c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-03T06:27:11.894 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:11.890+0000 7fabcc9982c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-03T06:27:11.903 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:11.895+0000 7f44dcc482c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-03T06:27:12.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:12.090+0000 7f8fef7fe700 1 -- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 msgr2=0x7f8fdc0673e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/32113 2023-03-03T06:27:12.094 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:12.090+0000 7f8fef7fe700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 0x7f8fdc0673e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-03-03T06:27:12.131 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:12.126+0000 7fabcc9982c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-03T06:27:12.143 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:12.135+0000 7f44dcc482c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-03T06:27:12.248 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:12.242+0000 7fabcc9982c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-03T06:27:12.260 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:12.255+0000 7f44dcc482c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-03T06:27:12.381 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:12.374+0000 7fabcc9982c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-03T06:27:12.395 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:12.387+0000 7f44dcc482c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-03T06:27:12.498 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:12.494+0000 7fabcc9982c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-03T06:27:12.515 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:12.507+0000 7f44dcc482c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-03T06:27:12.788 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:12.782+0000 7fabcc9982c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-03T06:27:12.806 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:12.799+0000 7f44dcc482c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-03T06:27:13.026 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:13.022+0000 7fabcc9982c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-03T06:27:13.054 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:13.047+0000 7f44dcc482c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-03T06:27:13.152 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:13.146+0000 7fabcc9982c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-03T06:27:13.177 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:13.171+0000 7f44dcc482c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-03T06:27:13.954 INFO:tasks.ceph.mgr.y.smithi003.stderr:2023-03-03T06:27:13.950+0000 7fabcc9982c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-03T06:27:13.978 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:13.971+0000 7f44dcc482c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-03T06:27:14.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:14.014+0000 7f8fecff9700 1 -- 172.21.15.3:0/1960315799 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 7) v1 ==== 83746+0+0 (secure 0 0 0) 0x7f8fe0010070 con 0x7f8ff00c9520 2023-03-03T06:27:14.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:14.018+0000 7f8fecff9700 1 -- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 msgr2=0x7f8fdc0673e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-03T06:27:14.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:14.018+0000 7f8fecff9700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 0x7f8fdc0673e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:14.315 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:14.307+0000 7f44a1d3c700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:27:14.316 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:14.307+0000 7f44a1d3c700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:27:14.316 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:14.307+0000 7f44a1d3c700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:27:14.316 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:14.307+0000 7f44a1d3c700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:27:14.316 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:14.307+0000 7f44a1d3c700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:27:14.317 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:14.311+0000 7f44a3d40700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:27:14.317 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:14.311+0000 7f44a3d40700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:27:14.317 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:14.311+0000 7f44a3d40700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:27:14.317 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:14.311+0000 7f44a3d40700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:27:14.318 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:27:14.311+0000 7f44a3d40700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:27:15.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.054+0000 7f8fecff9700 1 -- 172.21.15.3:0/1960315799 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mgrmap(e 8) v1 ==== 83856+0+0 (secure 0 0 0) 0x7f8fe004f780 con 0x7f8ff00c9520 2023-03-03T06:27:15.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.054+0000 7f8fecff9700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 0x7f8fdc0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:15.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.054+0000 7f8fecff9700 1 -- 172.21.15.3:0/1960315799 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ff015b780 con 0x7f8fdc064f30 2023-03-03T06:27:15.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.062+0000 7f8fef7fe700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 0x7f8fdc0673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:15.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.066+0000 7f8fef7fe700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 0x7f8fdc0673e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4019040 tx=0x7f8fe4008030 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:15.215 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.210+0000 7f8fecff9700 1 -- 172.21.15.3:0/1960315799 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f8ff015b780 con 0x7f8fdc064f30 2023-03-03T06:27:15.216 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path /snap_test_dir1/dir0 2023-03-03T06:27:15.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.214+0000 7f8ff5534700 1 -- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 msgr2=0x7f8fdc0673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:15.219 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.214+0000 7f8ff5534700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 0x7f8fdc0673e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4019040 tx=0x7f8fe4008030 comp rx=0 tx=0).stop 2023-03-03T06:27:15.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.214+0000 7f8ff5534700 1 -- 172.21.15.3:0/1960315799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00c9520 msgr2=0x7f8ff00c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:15.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.214+0000 7f8ff5534700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00c9520 0x7f8ff00c8350 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0001eb0 tx=0x7f8fe000c860 comp rx=0 tx=0).stop 2023-03-03T06:27:15.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.214+0000 7f8ff5534700 1 -- 172.21.15.3:0/1960315799 shutdown_connections 2023-03-03T06:27:15.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.214+0000 7f8ff5534700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f8fdc064f30 0x7f8fdc0673e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:15.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.214+0000 7f8ff5534700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ff0158810 0x7f8ff00aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:15.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.214+0000 7f8ff5534700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f8ff00ccc70 0x7f8ff00c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:15.220 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.214+0000 7f8ff5534700 1 --2- 172.21.15.3:0/1960315799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00c9520 0x7f8ff00c8350 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:15.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.214+0000 7f8ff5534700 1 -- 172.21.15.3:0/1960315799 >> 172.21.15.3:0/1960315799 conn(0x7f8ff00607c0 msgr2=0x7f8ff006cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:15.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.214+0000 7f8ff5534700 1 -- 172.21.15.3:0/1960315799 shutdown_connections 2023-03-03T06:27:15.221 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.214+0000 7f8ff5534700 1 -- 172.21.15.3:0/1960315799 wait complete. 2023-03-03T06:27:15.257 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-03-03T06:27:15.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:15.258 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-03-03T06:27:15.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir1 1M 2023-03-03T06:27:15.551 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.546+0000 7f31ad1b4700 1 Processor -- start 2023-03-03T06:27:15.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.546+0000 7f31ad1b4700 1 -- start start 2023-03-03T06:27:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31ad1b4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31a806a780 0x7f31a80cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31ad1b4700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f31a80cb9a0 0x7f31a80709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31ad1b4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a8071340 con 0x7f31a8070ef0 2023-03-03T06:27:15.554 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31ad1b4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a80caac0 con 0x7f31a806a780 2023-03-03T06:27:15.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31ad1b4700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a80cf5f0 con 0x7f31a80cb9a0 2023-03-03T06:27:15.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31a7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31a806a780 0x7f31a80cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:15.555 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31a7fff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f31a806a780 0x7f31a80cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48904/0 (socket says 172.21.15.3:48904) 2023-03-03T06:27:15.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31a7fff700 1 -- 172.21.15.3:0/3300626995 learned_addr learned my addr 172.21.15.3:0/3300626995 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:15.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31ac9b3700 1 --1- 172.21.15.3:0/3300626995 >> v1:172.21.15.3:6789/0 conn(0x7f31a8070ef0 0x7f31a806c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42642/0 (socket says 172.21.15.3:42642) 2023-03-03T06:27:15.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f319f7fe700 1 --2- 172.21.15.3:0/3300626995 >> v2:172.21.15.32:3300/0 conn(0x7f31a80cb9a0 0x7f31a80709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:15.556 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31a7fff700 1 -- 172.21.15.3:0/3300626995 >> v2:172.21.15.32:3300/0 conn(0x7f31a80cb9a0 msgr2=0x7f31a80709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:15.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31a7fff700 1 --2- 172.21.15.3:0/3300626995 >> v2:172.21.15.32:3300/0 conn(0x7f31a80cb9a0 0x7f31a80709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:15.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31a7fff700 1 -- 172.21.15.3:0/3300626995 >> v1:172.21.15.3:6789/0 conn(0x7f31a8070ef0 legacy=0x7f31a806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:15.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31a7fff700 1 -- 172.21.15.3:0/3300626995 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a815c230 con 0x7f31a806a780 2023-03-03T06:27:15.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31a7fff700 1 --2- 172.21.15.3:0/3300626995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a806a780 0x7f31a80cace0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f31a80b0900 tx=0x7f319800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a73db3e0eafcf5c server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:15.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31a77fe700 1 -- 172.21.15.3:0/3300626995 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198010070 con 0x7f31a806a780 2023-03-03T06:27:15.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31a77fe700 1 -- 172.21.15.3:0/3300626995 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319800db80 con 0x7f31a806a780 2023-03-03T06:27:15.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.550+0000 7f31a77fe700 1 -- 172.21.15.3:0/3300626995 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198016400 con 0x7f31a806a780 2023-03-03T06:27:15.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.554+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3300626995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a806a780 msgr2=0x7f31a80cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:15.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.554+0000 7f31ad1b4700 1 --2- 172.21.15.3:0/3300626995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a806a780 0x7f31a80cace0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f31a80b0900 tx=0x7f319800d430 comp rx=0 tx=0).stop 2023-03-03T06:27:15.559 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.554+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3300626995 shutdown_connections 2023-03-03T06:27:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.554+0000 7f31ad1b4700 1 --2- 172.21.15.3:0/3300626995 >> v2:172.21.15.32:3300/0 conn(0x7f31a80cb9a0 0x7f31a80709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.554+0000 7f31ad1b4700 1 --2- 172.21.15.3:0/3300626995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a806a780 0x7f31a80cace0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.554+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3300626995 >> 172.21.15.3:0/3300626995 conn(0x7f31a80607c0 msgr2=0x7f31a8060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.554+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3300626995 shutdown_connections 2023-03-03T06:27:15.560 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.554+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3300626995 wait complete. 2023-03-03T06:27:15.561 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f31ad1b4700 1 Processor -- start 2023-03-03T06:27:15.562 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f31ad1b4700 1 -- start start 2023-03-03T06:27:15.563 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f31ad1b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a806a780 0x7f31a8158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f31ad1b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a8070ef0 0x7f31a80b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f31ad1b4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a80cb9a0 0x7f31a80c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f31ad1b4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a8159350 con 0x7f31a8070ef0 2023-03-03T06:27:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f31ad1b4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a81ef7c0 con 0x7f31a806a780 2023-03-03T06:27:15.564 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f31ad1b4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80b4640 con 0x7f31a80cb9a0 2023-03-03T06:27:15.566 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f319f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a8070ef0 0x7f31a80b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:15.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f319f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a8070ef0 0x7f31a80b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48914/0 (socket says 172.21.15.3:48914) 2023-03-03T06:27:15.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f319f7fe700 1 -- 172.21.15.3:0/3583118327 learned_addr learned my addr 172.21.15.3:0/3583118327 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:15.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f31a7fff700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a806a780 0x7f31a8158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:15.567 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f31ac9b3700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a80cb9a0 0x7f31a80c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f319f7fe700 1 -- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a80cb9a0 msgr2=0x7f31a80c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f319f7fe700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a80cb9a0 0x7f31a80c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f319f7fe700 1 -- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a806a780 msgr2=0x7f31a8158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:15.568 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f319f7fe700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a806a780 0x7f31a8158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f319f7fe700 1 -- 172.21.15.3:0/3583118327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319800bd40 con 0x7f31a8070ef0 2023-03-03T06:27:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f31ac9b3700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a80cb9a0 0x7f31a80c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:27:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.558+0000 7f319f7fe700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a8070ef0 0x7f31a80b3b80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f319000e760 tx=0x7f319000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:15.569 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.562+0000 7f31a57fa700 1 -- 172.21.15.3:0/3583118327 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190013610 con 0x7f31a8070ef0 2023-03-03T06:27:15.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.562+0000 7f31a57fa700 1 -- 172.21.15.3:0/3583118327 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3190013770 con 0x7f31a8070ef0 2023-03-03T06:27:15.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.562+0000 7f31a57fa700 1 -- 172.21.15.3:0/3583118327 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190013ae0 con 0x7f31a8070ef0 2023-03-03T06:27:15.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.562+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3583118327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a80cbf60 con 0x7f31a8070ef0 2023-03-03T06:27:15.570 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.562+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3583118327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a80cd130 con 0x7f31a8070ef0 2023-03-03T06:27:15.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.562+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3583118327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a815c4f0 con 0x7f31a8070ef0 2023-03-03T06:27:15.571 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.562+0000 7f31a57fa700 1 -- 172.21.15.3:0/3583118327 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) v1 ==== 83856+0+0 (secure 0 0 0) 0x7f3190023070 con 0x7f31a8070ef0 2023-03-03T06:27:15.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.566+0000 7f31a57fa700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3188064e50 0x7f3188067300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:15.572 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.566+0000 7f31a57fa700 1 -- 172.21.15.3:0/3583118327 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f3190081d10 con 0x7f31a8070ef0 2023-03-03T06:27:15.573 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.566+0000 7f31a7fff700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3188064e50 0x7f3188067300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:15.574 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.570+0000 7f31a7fff700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3188064e50 0x7f3188067300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f319800d470 tx=0x7f3198012000 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:15.577 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.574+0000 7f31a57fa700 1 -- 172.21.15.3:0/3583118327 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f3190087020 con 0x7f31a8070ef0 2023-03-03T06:27:15.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.766+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3583118327 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f31a8073e90 con 0x7f3188064e50 2023-03-03T06:27:15.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.894+0000 7f31a57fa700 1 -- 172.21.15.3:0/3583118327 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f31a8073e90 con 0x7f3188064e50 2023-03-03T06:27:15.899 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path /snap_test_dir1/dir1 2023-03-03T06:27:15.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.898+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3188064e50 msgr2=0x7f3188067300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:15.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.898+0000 7f31ad1b4700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3188064e50 0x7f3188067300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f319800d470 tx=0x7f3198012000 comp rx=0 tx=0).stop 2023-03-03T06:27:15.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.898+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3583118327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a8070ef0 msgr2=0x7f31a80b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:15.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.898+0000 7f31ad1b4700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a8070ef0 0x7f31a80b3b80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f319000e760 tx=0x7f319000eb20 comp rx=0 tx=0).stop 2023-03-03T06:27:15.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.898+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3583118327 shutdown_connections 2023-03-03T06:27:15.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.898+0000 7f31ad1b4700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3188064e50 0x7f3188067300 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:15.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.898+0000 7f31ad1b4700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f31a80cb9a0 0x7f31a80c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:15.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.898+0000 7f31ad1b4700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31a8070ef0 0x7f31a80b3b80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:15.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.898+0000 7f31ad1b4700 1 --2- 172.21.15.3:0/3583118327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a806a780 0x7f31a8158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:15.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.898+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3583118327 >> 172.21.15.3:0/3583118327 conn(0x7f31a80607c0 msgr2=0x7f31a80ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:15.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.898+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3583118327 shutdown_connections 2023-03-03T06:27:15.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:15.898+0000 7f31ad1b4700 1 -- 172.21.15.3:0/3583118327 wait complete. 2023-03-03T06:27:15.941 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-03-03T06:27:15.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:15.942 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-03-03T06:27:15.975 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir2 1M 2023-03-03T06:27:16.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.234+0000 7fe5c5ce8700 1 Processor -- start 2023-03-03T06:27:16.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.234+0000 7fe5c5ce8700 1 -- start start 2023-03-03T06:27:16.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c5ce8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5c01591d0 0x7fe5c00cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:16.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c5ce8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5c00b07c0 0x7fe5c00b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:16.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c5ce8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5c015cc70 0x7fe5c015d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:16.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c5ce8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c01595a0 con 0x7fe5c01591d0 2023-03-03T06:27:16.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c5ce8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c00b1180 con 0x7fe5c00b07c0 2023-03-03T06:27:16.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c5ce8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c00ce810 con 0x7fe5c015cc70 2023-03-03T06:27:16.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c4ce6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5c01591d0 0x7fe5c00cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:16.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c4ce6700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5c01591d0 0x7fe5c00cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48924/0 (socket says 172.21.15.3:48924) 2023-03-03T06:27:16.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c4ce6700 1 -- 172.21.15.3:0/1912353784 learned_addr learned my addr 172.21.15.3:0/1912353784 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:16.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5bffff700 1 --2- 172.21.15.3:0/1912353784 >> v2:172.21.15.32:3300/0 conn(0x7fe5c00b07c0 0x7fe5c00b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:16.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c54e7700 1 --2- 172.21.15.3:0/1912353784 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5c015cc70 0x7fe5c015d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:16.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c4ce6700 1 -- 172.21.15.3:0/1912353784 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5c015cc70 msgr2=0x7fe5c015d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:16.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c4ce6700 1 --2- 172.21.15.3:0/1912353784 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5c015cc70 0x7fe5c015d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c4ce6700 1 -- 172.21.15.3:0/1912353784 >> v2:172.21.15.32:3300/0 conn(0x7fe5c00b07c0 msgr2=0x7fe5c00b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:16.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c4ce6700 1 --2- 172.21.15.3:0/1912353784 >> v2:172.21.15.32:3300/0 conn(0x7fe5c00b07c0 0x7fe5c00b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c4ce6700 1 -- 172.21.15.3:0/1912353784 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c00ceaf0 con 0x7fe5c01591d0 2023-03-03T06:27:16.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5bffff700 1 --2- 172.21.15.3:0/1912353784 >> v2:172.21.15.32:3300/0 conn(0x7fe5c00b07c0 0x7fe5c00b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:27:16.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.238+0000 7fe5c4ce6700 1 --2- 172.21.15.3:0/1912353784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c01591d0 0x7fe5c00cd6d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe5c006aab0 tx=0x7fe5b4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf81b0a95914135 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:16.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.242+0000 7fe5bf7fe700 1 -- 172.21.15.3:0/1912353784 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4011830 con 0x7fe5c01591d0 2023-03-03T06:27:16.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.242+0000 7fe5bf7fe700 1 -- 172.21.15.3:0/1912353784 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b4011990 con 0x7fe5c01591d0 2023-03-03T06:27:16.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.242+0000 7fe5bf7fe700 1 -- 172.21.15.3:0/1912353784 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4011d20 con 0x7fe5c01591d0 2023-03-03T06:27:16.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.242+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/1912353784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c01591d0 msgr2=0x7fe5c00cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:16.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.242+0000 7fe5c5ce8700 1 --2- 172.21.15.3:0/1912353784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c01591d0 0x7fe5c00cd6d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe5c006aab0 tx=0x7fe5b4009bf0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.242+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/1912353784 shutdown_connections 2023-03-03T06:27:16.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.242+0000 7fe5c5ce8700 1 --2- 172.21.15.3:0/1912353784 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5c015cc70 0x7fe5c015d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.242+0000 7fe5c5ce8700 1 --2- 172.21.15.3:0/1912353784 >> v2:172.21.15.32:3300/0 conn(0x7fe5c00b07c0 0x7fe5c00b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.242+0000 7fe5c5ce8700 1 --2- 172.21.15.3:0/1912353784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c01591d0 0x7fe5c00cd6d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.242+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/1912353784 >> 172.21.15.3:0/1912353784 conn(0x7fe5c00607c0 msgr2=0x7fe5c0060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:16.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.242+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/1912353784 shutdown_connections 2023-03-03T06:27:16.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.242+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/1912353784 wait complete. 2023-03-03T06:27:16.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c5ce8700 1 Processor -- start 2023-03-03T06:27:16.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c5ce8700 1 -- start start 2023-03-03T06:27:16.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c5ce8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c00b07c0 0x7fe5c0164000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:16.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c5ce8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c01591d0 0x7fe5c014a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:16.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c5ce8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5c015cc70 0x7fe5c01584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:16.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c5ce8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c014ab10 con 0x7fe5c00b07c0 2023-03-03T06:27:16.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c5ce8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c00b3c30 con 0x7fe5c01591d0 2023-03-03T06:27:16.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c5ce8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c0162240 con 0x7fe5c015cc70 2023-03-03T06:27:16.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c4ce6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c00b07c0 0x7fe5c0164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:16.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c4ce6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c00b07c0 0x7fe5c0164000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48940/0 (socket says 172.21.15.3:48940) 2023-03-03T06:27:16.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c4ce6700 1 -- 172.21.15.3:0/3910511294 learned_addr learned my addr 172.21.15.3:0/3910511294 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:16.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c54e7700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5c015cc70 0x7fe5c01584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:16.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5bffff700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c01591d0 0x7fe5c014a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:16.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c4ce6700 1 -- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5c015cc70 msgr2=0x7fe5c01584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:16.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c4ce6700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5c015cc70 0x7fe5c01584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c4ce6700 1 -- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c01591d0 msgr2=0x7fe5c014a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:16.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c4ce6700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c01591d0 0x7fe5c014a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5c4ce6700 1 -- 172.21.15.3:0/3910511294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b4009570 con 0x7fe5c00b07c0 2023-03-03T06:27:16.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.246+0000 7fe5bffff700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c01591d0 0x7fe5c014a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:27:16.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.250+0000 7fe5c4ce6700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c00b07c0 0x7fe5c0164000 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009f40 tx=0x7fe5b4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:16.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.250+0000 7fe5bd7fa700 1 -- 172.21.15.3:0/3910511294 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4004240 con 0x7fe5c00b07c0 2023-03-03T06:27:16.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.250+0000 7fe5bd7fa700 1 -- 172.21.15.3:0/3910511294 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b401f070 con 0x7fe5c00b07c0 2023-03-03T06:27:16.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.250+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/3910511294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c00cf0b0 con 0x7fe5c00b07c0 2023-03-03T06:27:16.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.250+0000 7fe5bd7fa700 1 -- 172.21.15.3:0/3910511294 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4004600 con 0x7fe5c00b07c0 2023-03-03T06:27:16.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.250+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/3910511294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c00cedd0 con 0x7fe5c00b07c0 2023-03-03T06:27:16.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.250+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/3910511294 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c00cf390 con 0x7fe5c00b07c0 2023-03-03T06:27:16.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.254+0000 7fe5bd7fa700 1 -- 172.21.15.3:0/3910511294 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7fe5b400f050 con 0x7fe5c00b07c0 2023-03-03T06:27:16.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.254+0000 7fe5bd7fa700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe5a4065000 0x7fe5a40674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:16.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.254+0000 7fe5bd7fa700 1 -- 172.21.15.3:0/3910511294 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fe5b40834e0 con 0x7fe5c00b07c0 2023-03-03T06:27:16.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.254+0000 7fe5bffff700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe5a4065000 0x7fe5a40674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:16.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.258+0000 7fe5bffff700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe5a4065000 0x7fe5a40674b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe5c014c010 tx=0x7fe5ac008040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:16.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.258+0000 7fe5bd7fa700 1 -- 172.21.15.3:0/3910511294 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe5b4051360 con 0x7fe5c00b07c0 2023-03-03T06:27:16.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.478+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/3910511294 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5c0162380 con 0x7fe5a4065000 2023-03-03T06:27:16.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.602+0000 7fe5bd7fa700 1 -- 172.21.15.3:0/3910511294 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe5ac003df0 con 0x7fe5a4065000 2023-03-03T06:27:16.608 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path /snap_test_dir1/dir2 2023-03-03T06:27:16.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.606+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe5a4065000 msgr2=0x7fe5a40674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:16.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.606+0000 7fe5c5ce8700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe5a4065000 0x7fe5a40674b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe5c014c010 tx=0x7fe5ac008040 comp rx=0 tx=0).stop 2023-03-03T06:27:16.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.606+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/3910511294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c00b07c0 msgr2=0x7fe5c0164000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:16.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.606+0000 7fe5c5ce8700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c00b07c0 0x7fe5c0164000 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009f40 tx=0x7fe5b4004770 comp rx=0 tx=0).stop 2023-03-03T06:27:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.610+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/3910511294 shutdown_connections 2023-03-03T06:27:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.610+0000 7fe5c5ce8700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fe5a4065000 0x7fe5a40674b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.610+0000 7fe5c5ce8700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fe5c015cc70 0x7fe5c01584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.610+0000 7fe5c5ce8700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5c01591d0 0x7fe5c014a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.610+0000 7fe5c5ce8700 1 --2- 172.21.15.3:0/3910511294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5c00b07c0 0x7fe5c0164000 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.610+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/3910511294 >> 172.21.15.3:0/3910511294 conn(0x7fe5c00607c0 msgr2=0x7fe5c00ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.610+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/3910511294 shutdown_connections 2023-03-03T06:27:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.610+0000 7fe5c5ce8700 1 -- 172.21.15.3:0/3910511294 wait complete. 2023-03-03T06:27:16.652 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-03-03T06:27:16.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:16.653 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-03-03T06:27:16.682 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir3 1M 2023-03-03T06:27:16.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.942+0000 7f0ed134a700 1 Processor -- start 2023-03-03T06:27:16.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.942+0000 7f0ed134a700 1 -- start start 2023-03-03T06:27:16.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.946+0000 7f0ed134a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ecc0cad60 0x7f0ecc0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:16.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.946+0000 7f0ed134a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc0cab60 con 0x7f0ecc06f970 2023-03-03T06:27:16.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.946+0000 7f0ed134a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc06f4e0 con 0x7f0ecc158a00 2023-03-03T06:27:16.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.946+0000 7f0ed134a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc06ef00 con 0x7f0ecc0cad60 2023-03-03T06:27:16.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.946+0000 7f0ecbfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0ecc06f970 0x7f0ecc06b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42656/0 (socket says 172.21.15.3:42656) 2023-03-03T06:27:16.958 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.946+0000 7f0ecbfff700 1 -- 172.21.15.3:0/3876159718 learned_addr learned my addr 172.21.15.3:0/3876159718 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:16.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.950+0000 7f0eca7fc700 1 --2- 172.21.15.3:0/3876159718 >> v2:172.21.15.32:3300/0 conn(0x7f0ecc0cad60 0x7f0ecc0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:16.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.950+0000 7f0ec9ffb700 1 -- 172.21.15.3:0/3876159718 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801158810 0 0) 0x7f0ecc0cab60 con 0x7f0ecc06f970 2023-03-03T06:27:16.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.950+0000 7f0ec9ffb700 1 -- 172.21.15.3:0/3876159718 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eb40035d0 con 0x7f0ecc06f970 2023-03-03T06:27:16.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.950+0000 7f0ec9ffb700 1 -- 172.21.15.3:0/3876159718 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043498978 0 0) 0x7f0ecc06f4e0 con 0x7f0ecc158a00 2023-03-03T06:27:16.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.950+0000 7f0ec9ffb700 1 -- 172.21.15.3:0/3876159718 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ecc0cab60 con 0x7f0ecc158a00 2023-03-03T06:27:16.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.950+0000 7f0eca7fc700 1 -- 172.21.15.3:0/3876159718 >> v1:172.21.15.32:6789/0 conn(0x7f0ecc158a00 legacy=0x7f0ecc0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:16.959 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.950+0000 7f0eca7fc700 1 -- 172.21.15.3:0/3876159718 >> v1:172.21.15.3:6789/0 conn(0x7f0ecc06f970 legacy=0x7f0ecc06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:16.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.950+0000 7f0eca7fc700 1 -- 172.21.15.3:0/3876159718 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc070870 con 0x7f0ecc0cad60 2023-03-03T06:27:16.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.950+0000 7f0eca7fc700 1 --2- 172.21.15.3:0/3876159718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ecc0cad60 0x7f0ecc0b2540 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0009850 tx=0x7f0ec0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fae5bc3a15d9ddf server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:16.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.950+0000 7f0ec9ffb700 1 -- 172.21.15.3:0/3876159718 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec000c880 con 0x7f0ecc0cad60 2023-03-03T06:27:16.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.950+0000 7f0ec9ffb700 1 -- 172.21.15.3:0/3876159718 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec000c9e0 con 0x7f0ecc0cad60 2023-03-03T06:27:16.960 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.950+0000 7f0ec9ffb700 1 -- 172.21.15.3:0/3876159718 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec000cd90 con 0x7f0ecc0cad60 2023-03-03T06:27:16.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.954+0000 7f0ed134a700 1 -- 172.21.15.3:0/3876159718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ecc0cad60 msgr2=0x7f0ecc0b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:16.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.954+0000 7f0ed134a700 1 --2- 172.21.15.3:0/3876159718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ecc0cad60 0x7f0ecc0b2540 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0009850 tx=0x7f0ec0009b60 comp rx=0 tx=0).stop 2023-03-03T06:27:16.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.954+0000 7f0ed134a700 1 -- 172.21.15.3:0/3876159718 shutdown_connections 2023-03-03T06:27:16.961 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.954+0000 7f0ed134a700 1 --2- 172.21.15.3:0/3876159718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ecc0cad60 0x7f0ecc0b2540 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.954+0000 7f0ed134a700 1 -- 172.21.15.3:0/3876159718 >> 172.21.15.3:0/3876159718 conn(0x7f0ecc0ccc40 msgr2=0x7f0ecc0cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:16.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.954+0000 7f0ed134a700 1 -- 172.21.15.3:0/3876159718 shutdown_connections 2023-03-03T06:27:16.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.954+0000 7f0ed134a700 1 -- 172.21.15.3:0/3876159718 wait complete. 2023-03-03T06:27:16.962 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.958+0000 7f0ed134a700 1 Processor -- start 2023-03-03T06:27:16.963 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.958+0000 7f0ed134a700 1 -- start start 2023-03-03T06:27:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.958+0000 7f0ed134a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ecc06f970 0x7f0ecc148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:16.964 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.958+0000 7f0ed134a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ecc0cad60 0x7f0ecc147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.958+0000 7f0ed134a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ecc158a00 0x7f0ecc0cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.958+0000 7f0ed134a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0ca230 con 0x7f0ecc158a00 2023-03-03T06:27:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.958+0000 7f0ed134a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0c9f50 con 0x7f0ecc0cad60 2023-03-03T06:27:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.958+0000 7f0ed134a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0c7430 con 0x7f0ecc06f970 2023-03-03T06:27:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.958+0000 7f0ecbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ecc158a00 0x7f0ecc0cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.958+0000 7f0ecbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ecc158a00 0x7f0ecc0cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48954/0 (socket says 172.21.15.3:48954) 2023-03-03T06:27:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.958+0000 7f0ecbfff700 1 -- 172.21.15.3:0/1861338518 learned_addr learned my addr 172.21.15.3:0/1861338518 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.958+0000 7f0eca7fc700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ecc0cad60 0x7f0ecc147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.962+0000 7f0ecaffd700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ecc06f970 0x7f0ecc148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.962+0000 7f0ecbfff700 1 -- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ecc06f970 msgr2=0x7f0ecc148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.962+0000 7f0ecbfff700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ecc06f970 0x7f0ecc148be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.962+0000 7f0ecbfff700 1 -- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ecc0cad60 msgr2=0x7f0ecc147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.962+0000 7f0ecbfff700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ecc0cad60 0x7f0ecc147960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.962+0000 7f0ecbfff700 1 -- 172.21.15.3:0/1861338518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec00094b0 con 0x7f0ecc158a00 2023-03-03T06:27:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.962+0000 7f0ecbfff700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ecc158a00 0x7f0ecc0cf100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400caf0 tx=0x7f0ec400cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.962+0000 7f0eb37fe700 1 -- 172.21.15.3:0/1861338518 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400a770 con 0x7f0ecc158a00 2023-03-03T06:27:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.962+0000 7f0eb37fe700 1 -- 172.21.15.3:0/1861338518 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec400a8d0 con 0x7f0ecc158a00 2023-03-03T06:27:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.962+0000 7f0ed134a700 1 -- 172.21.15.3:0/1861338518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc0ce4a0 con 0x7f0ecc158a00 2023-03-03T06:27:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.962+0000 7f0eb37fe700 1 -- 172.21.15.3:0/1861338518 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400ac60 con 0x7f0ecc158a00 2023-03-03T06:27:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.962+0000 7f0ed134a700 1 -- 172.21.15.3:0/1861338518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc06f170 con 0x7f0ecc158a00 2023-03-03T06:27:16.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.966+0000 7f0ed134a700 1 -- 172.21.15.3:0/1861338518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc158dd0 con 0x7f0ecc158a00 2023-03-03T06:27:16.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.966+0000 7f0eb37fe700 1 -- 172.21.15.3:0/1861338518 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f0ec4019030 con 0x7f0ecc158a00 2023-03-03T06:27:16.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.966+0000 7f0eb37fe700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0eb8065000 0x7f0eb80674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:16.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.966+0000 7f0eb37fe700 1 -- 172.21.15.3:0/1861338518 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f0ec4081bd0 con 0x7f0ecc158a00 2023-03-03T06:27:16.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.970+0000 7f0ecaffd700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0eb8065000 0x7f0eb80674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:16.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.970+0000 7f0ecaffd700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0eb8065000 0x7f0eb80674b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc001250 tx=0x7f0ebc009040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:16.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:16.974+0000 7f0eb37fe700 1 -- 172.21.15.3:0/1861338518 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f0ec404f910 con 0x7f0ecc158a00 2023-03-03T06:27:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.166+0000 7f0ed134a700 1 -- 172.21.15.3:0/1861338518 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ecc0c7290 con 0x7f0eb8065000 2023-03-03T06:27:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.294+0000 7f0eb37fe700 1 -- 172.21.15.3:0/1861338518 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f0ecc0c7290 con 0x7f0eb8065000 2023-03-03T06:27:17.300 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path /snap_test_dir1/dir3 2023-03-03T06:27:17.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.298+0000 7f0ed134a700 1 -- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0eb8065000 msgr2=0x7f0eb80674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:17.304 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.298+0000 7f0ed134a700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0eb8065000 0x7f0eb80674b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc001250 tx=0x7f0ebc009040 comp rx=0 tx=0).stop 2023-03-03T06:27:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.298+0000 7f0ed134a700 1 -- 172.21.15.3:0/1861338518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ecc158a00 msgr2=0x7f0ecc0cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.298+0000 7f0ed134a700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ecc158a00 0x7f0ecc0cf100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400caf0 tx=0x7f0ec400cb20 comp rx=0 tx=0).stop 2023-03-03T06:27:17.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.298+0000 7f0ed134a700 1 -- 172.21.15.3:0/1861338518 shutdown_connections 2023-03-03T06:27:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.298+0000 7f0ed134a700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0eb8065000 0x7f0eb80674b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.298+0000 7f0ed134a700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ecc158a00 0x7f0ecc0cf100 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.298+0000 7f0ed134a700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ecc0cad60 0x7f0ecc147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.298+0000 7f0ed134a700 1 --2- 172.21.15.3:0/1861338518 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0ecc06f970 0x7f0ecc148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.298+0000 7f0ed134a700 1 -- 172.21.15.3:0/1861338518 >> 172.21.15.3:0/1861338518 conn(0x7f0ecc0ccc40 msgr2=0x7f0ecc0b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.298+0000 7f0ed134a700 1 -- 172.21.15.3:0/1861338518 shutdown_connections 2023-03-03T06:27:17.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.298+0000 7f0ed134a700 1 -- 172.21.15.3:0/1861338518 wait complete. 2023-03-03T06:27:17.335 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-03-03T06:27:17.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:17.337 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-03-03T06:27:17.368 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir4 1M 2023-03-03T06:27:17.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.614+0000 7fa3be597700 1 Processor -- start 2023-03-03T06:27:17.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.618+0000 7fa3be597700 1 -- start start 2023-03-03T06:27:17.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.618+0000 7fa3be597700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3b8156ba0 0x7fa3b8071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:17.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3be597700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3b80d0b30 0x7fa3b80d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:17.626 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3be597700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b80cf3b0 con 0x7fa3b815d6e0 2023-03-03T06:27:17.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3bd595700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3b8156ba0 0x7fa3b8071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:17.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3be597700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b80d14f0 con 0x7fa3b8156ba0 2023-03-03T06:27:17.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3be597700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b8156f70 con 0x7fa3b80d0b30 2023-03-03T06:27:17.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3bd595700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3b8156ba0 0x7fa3b8071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48968/0 (socket says 172.21.15.3:48968) 2023-03-03T06:27:17.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3bd595700 1 -- 172.21.15.3:0/1466227287 learned_addr learned my addr 172.21.15.3:0/1466227287 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:17.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3bd595700 1 -- 172.21.15.3:0/1466227287 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3b80d0b30 msgr2=0x7fa3b80d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:17.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3bd595700 1 --2- 172.21.15.3:0/1466227287 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3b80d0b30 0x7fa3b80d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3bd595700 1 -- 172.21.15.3:0/1466227287 >> v1:172.21.15.32:6789/0 conn(0x7fa3b815d6e0 legacy=0x7fa3b80cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:17.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3bd595700 1 -- 172.21.15.3:0/1466227287 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b80cd320 con 0x7fa3b8156ba0 2023-03-03T06:27:17.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3bd595700 1 --2- 172.21.15.3:0/1466227287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b8156ba0 0x7fa3b8071280 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009e50 tx=0x7fa3b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f347207fbb8d75 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:17.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3affff700 1 -- 172.21.15.3:0/1466227287 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b400e070 con 0x7fa3b8156ba0 2023-03-03T06:27:17.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3affff700 1 -- 172.21.15.3:0/1466227287 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b4003a40 con 0x7fa3b8156ba0 2023-03-03T06:27:17.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3affff700 1 -- 172.21.15.3:0/1466227287 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4003df0 con 0x7fa3b8156ba0 2023-03-03T06:27:17.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3be597700 1 -- 172.21.15.3:0/1466227287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b8156ba0 msgr2=0x7fa3b8071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:17.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.622+0000 7fa3be597700 1 --2- 172.21.15.3:0/1466227287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b8156ba0 0x7fa3b8071280 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009e50 tx=0x7fa3b400b330 comp rx=0 tx=0).stop 2023-03-03T06:27:17.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.626+0000 7fa3be597700 1 -- 172.21.15.3:0/1466227287 shutdown_connections 2023-03-03T06:27:17.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.626+0000 7fa3be597700 1 --2- 172.21.15.3:0/1466227287 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3b80d0b30 0x7fa3b80d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.626+0000 7fa3be597700 1 --2- 172.21.15.3:0/1466227287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b8156ba0 0x7fa3b8071280 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.626+0000 7fa3be597700 1 -- 172.21.15.3:0/1466227287 >> 172.21.15.3:0/1466227287 conn(0x7fa3b806e5c0 msgr2=0x7fa3b806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:17.631 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.626+0000 7fa3be597700 1 -- 172.21.15.3:0/1466227287 shutdown_connections 2023-03-03T06:27:17.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.626+0000 7fa3be597700 1 -- 172.21.15.3:0/1466227287 wait complete. 2023-03-03T06:27:17.633 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.626+0000 7fa3be597700 1 Processor -- start 2023-03-03T06:27:17.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3be597700 1 -- start start 2023-03-03T06:27:17.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3be597700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b80d0b30 0x7fa3b81ef510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:17.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3be597700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b8156ba0 0x7fa3b8158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:17.635 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3be597700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3b815d6e0 0x7fa3b8158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:17.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3be597700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80b70f0 con 0x7fa3b80d0b30 2023-03-03T06:27:17.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3be597700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b81f5b40 con 0x7fa3b8156ba0 2023-03-03T06:27:17.636 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3be597700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80b6850 con 0x7fa3b815d6e0 2023-03-03T06:27:17.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3bd595700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b80d0b30 0x7fa3b81ef510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:17.637 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3bd595700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b80d0b30 0x7fa3b81ef510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48976/0 (socket says 172.21.15.3:48976) 2023-03-03T06:27:17.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3bd595700 1 -- 172.21.15.3:0/2203983829 learned_addr learned my addr 172.21.15.3:0/2203983829 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:17.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3bdd96700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3b815d6e0 0x7fa3b8158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:17.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3bcd94700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b8156ba0 0x7fa3b8158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:17.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3bd595700 1 -- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3b815d6e0 msgr2=0x7fa3b8158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:17.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3bd595700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3b815d6e0 0x7fa3b8158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3bd595700 1 -- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b8156ba0 msgr2=0x7fa3b8158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:17.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3bd595700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b8156ba0 0x7fa3b8158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3bd595700 1 -- 172.21.15.3:0/2203983829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b4009b30 con 0x7fa3b80d0b30 2023-03-03T06:27:17.639 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.630+0000 7fa3bcd94700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b8156ba0 0x7fa3b8158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:27:17.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.634+0000 7fa3bd595700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b80d0b30 0x7fa3b81ef510 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4015040 tx=0x7fa3b4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:17.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.634+0000 7fa3adffb700 1 -- 172.21.15.3:0/2203983829 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4004240 con 0x7fa3b80d0b30 2023-03-03T06:27:17.640 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.634+0000 7fa3adffb700 1 -- 172.21.15.3:0/2203983829 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b40043a0 con 0x7fa3b80d0b30 2023-03-03T06:27:17.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.634+0000 7fa3adffb700 1 -- 172.21.15.3:0/2203983829 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4004750 con 0x7fa3b80d0b30 2023-03-03T06:27:17.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.634+0000 7fa3be597700 1 -- 172.21.15.3:0/2203983829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b80cd8e0 con 0x7fa3b80d0b30 2023-03-03T06:27:17.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.634+0000 7fa3be597700 1 -- 172.21.15.3:0/2203983829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b80cd600 con 0x7fa3b80d0b30 2023-03-03T06:27:17.641 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.634+0000 7fa3be597700 1 -- 172.21.15.3:0/2203983829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b80717c0 con 0x7fa3b80d0b30 2023-03-03T06:27:17.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.638+0000 7fa3adffb700 1 -- 172.21.15.3:0/2203983829 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7fa3b4012430 con 0x7fa3b80d0b30 2023-03-03T06:27:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.638+0000 7fa3adffb700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa39c065000 0x7fa39c0674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.638+0000 7fa3adffb700 1 -- 172.21.15.3:0/2203983829 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fa3b4083890 con 0x7fa3b80d0b30 2023-03-03T06:27:17.645 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.642+0000 7fa3bcd94700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa39c065000 0x7fa39c0674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:17.647 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.642+0000 7fa3bcd94700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa39c065000 0x7fa39c0674b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80b5700 tx=0x7fa3a8009040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:17.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.646+0000 7fa3adffb700 1 -- 172.21.15.3:0/2203983829 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa3b4051570 con 0x7fa3b80d0b30 2023-03-03T06:27:17.845 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.838+0000 7fa3be597700 1 -- 172.21.15.3:0/2203983829 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3b8073e90 con 0x7fa39c065000 2023-03-03T06:27:17.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.962+0000 7fa3adffb700 1 -- 172.21.15.3:0/2203983829 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa3b8073e90 con 0x7fa39c065000 2023-03-03T06:27:17.969 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path /snap_test_dir1/dir4 2023-03-03T06:27:17.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.970+0000 7fa3be597700 1 -- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa39c065000 msgr2=0x7fa39c0674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:17.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.970+0000 7fa3be597700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa39c065000 0x7fa39c0674b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80b5700 tx=0x7fa3a8009040 comp rx=0 tx=0).stop 2023-03-03T06:27:17.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.970+0000 7fa3be597700 1 -- 172.21.15.3:0/2203983829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b80d0b30 msgr2=0x7fa3b81ef510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:17.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.970+0000 7fa3be597700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b80d0b30 0x7fa3b81ef510 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4015040 tx=0x7fa3b4004af0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.970+0000 7fa3be597700 1 -- 172.21.15.3:0/2203983829 shutdown_connections 2023-03-03T06:27:17.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.970+0000 7fa3be597700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa39c065000 0x7fa39c0674b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.970+0000 7fa3be597700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa3b815d6e0 0x7fa3b8158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.970+0000 7fa3be597700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3b8156ba0 0x7fa3b8158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.970+0000 7fa3be597700 1 --2- 172.21.15.3:0/2203983829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3b80d0b30 0x7fa3b81ef510 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:17.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.970+0000 7fa3be597700 1 -- 172.21.15.3:0/2203983829 >> 172.21.15.3:0/2203983829 conn(0x7fa3b806e5c0 msgr2=0x7fa3b815f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:17.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.970+0000 7fa3be597700 1 -- 172.21.15.3:0/2203983829 shutdown_connections 2023-03-03T06:27:17.976 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:17.970+0000 7fa3be597700 1 -- 172.21.15.3:0/2203983829 wait complete. 2023-03-03T06:27:18.012 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-03-03T06:27:18.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:18.012 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-03-03T06:27:18.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir5 1M 2023-03-03T06:27:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.298+0000 7f4af739a700 1 Processor -- start 2023-03-03T06:27:18.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.298+0000 7f4af739a700 1 -- start start 2023-03-03T06:27:18.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.298+0000 7f4af739a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4af00cb9a0 0x7f4af00cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:18.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.298+0000 7f4af739a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af015c230 con 0x7f4af006a780 2023-03-03T06:27:18.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.298+0000 7f4af739a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af0158b10 con 0x7f4af00709b0 2023-03-03T06:27:18.305 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.298+0000 7f4af739a700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af00caac0 con 0x7f4af00cb9a0 2023-03-03T06:27:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af6398700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4af006a780 0x7f4af0158400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42664/0 (socket says 172.21.15.3:42664) 2023-03-03T06:27:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af6398700 1 -- 172.21.15.3:0/3374759445 learned_addr learned my addr 172.21.15.3:0/3374759445 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af5b97700 1 --2- 172.21.15.3:0/3374759445 >> v2:172.21.15.32:3300/0 conn(0x7f4af00cb9a0 0x7f4af00cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:18.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af5396700 1 -- 172.21.15.3:0/3374759445 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339236674 0 0) 0x7f4af015c230 con 0x7f4af006a780 2023-03-03T06:27:18.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af5396700 1 -- 172.21.15.3:0/3374759445 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4adc0035d0 con 0x7f4af006a780 2023-03-03T06:27:18.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af5396700 1 -- 172.21.15.3:0/3374759445 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663773455 0 0) 0x7f4af0158b10 con 0x7f4af00709b0 2023-03-03T06:27:18.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af5396700 1 -- 172.21.15.3:0/3374759445 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4af015c230 con 0x7f4af00709b0 2023-03-03T06:27:18.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af5b97700 1 -- 172.21.15.3:0/3374759445 >> v1:172.21.15.32:6789/0 conn(0x7f4af00709b0 legacy=0x7f4af0163560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:18.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af5b97700 1 -- 172.21.15.3:0/3374759445 >> v1:172.21.15.3:6789/0 conn(0x7f4af006a780 legacy=0x7f4af0158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:18.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af5b97700 1 -- 172.21.15.3:0/3374759445 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af00cd130 con 0x7f4af00cb9a0 2023-03-03T06:27:18.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af5b97700 1 --2- 172.21.15.3:0/3374759445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af00cb9a0 0x7f4af00cace0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009850 tx=0x7f4ae0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4941e165e39b0f5 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:18.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af5396700 1 -- 172.21.15.3:0/3374759445 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0011830 con 0x7f4af00cb9a0 2023-03-03T06:27:18.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af5396700 1 -- 172.21.15.3:0/3374759445 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae0011990 con 0x7f4af00cb9a0 2023-03-03T06:27:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af5396700 1 -- 172.21.15.3:0/3374759445 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0011d40 con 0x7f4af00cb9a0 2023-03-03T06:27:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af739a700 1 -- 172.21.15.3:0/3374759445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af00cb9a0 msgr2=0x7f4af00cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.302+0000 7f4af739a700 1 --2- 172.21.15.3:0/3374759445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af00cb9a0 0x7f4af00cace0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009850 tx=0x7f4ae0009b60 comp rx=0 tx=0).stop 2023-03-03T06:27:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.306+0000 7f4af739a700 1 -- 172.21.15.3:0/3374759445 shutdown_connections 2023-03-03T06:27:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.306+0000 7f4af739a700 1 --2- 172.21.15.3:0/3374759445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af00cb9a0 0x7f4af00cace0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.306+0000 7f4af739a700 1 -- 172.21.15.3:0/3374759445 >> 172.21.15.3:0/3374759445 conn(0x7f4af00607c0 msgr2=0x7f4af0060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.306+0000 7f4af739a700 1 -- 172.21.15.3:0/3374759445 shutdown_connections 2023-03-03T06:27:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.306+0000 7f4af739a700 1 -- 172.21.15.3:0/3374759445 wait complete. 2023-03-03T06:27:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.310+0000 7f4af739a700 1 Processor -- start 2023-03-03T06:27:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.310+0000 7f4af739a700 1 -- start start 2023-03-03T06:27:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.310+0000 7f4af739a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af006a780 0x7f4af00c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:18.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.310+0000 7f4af739a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af00709b0 0x7f4af0152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:18.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.310+0000 7f4af739a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4af00cb9a0 0x7f4af00b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:18.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.310+0000 7f4af739a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00b16a0 con 0x7f4af006a780 2023-03-03T06:27:18.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.310+0000 7f4af739a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00b2500 con 0x7f4af00709b0 2023-03-03T06:27:18.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.310+0000 7f4af739a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00c8b70 con 0x7f4af00cb9a0 2023-03-03T06:27:18.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.310+0000 7f4af6398700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af006a780 0x7f4af00c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:18.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.310+0000 7f4af6398700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af006a780 0x7f4af00c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48980/0 (socket says 172.21.15.3:48980) 2023-03-03T06:27:18.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.310+0000 7f4af6398700 1 -- 172.21.15.3:0/1557170335 learned_addr learned my addr 172.21.15.3:0/1557170335 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:18.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4af6b99700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4af00cb9a0 0x7f4af00b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:18.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4af5b97700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af00709b0 0x7f4af0152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:18.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4af6398700 1 -- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4af00cb9a0 msgr2=0x7f4af00b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:18.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4af6398700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4af00cb9a0 0x7f4af00b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:18.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4af6398700 1 -- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af00709b0 msgr2=0x7f4af0152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:18.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4af6398700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af00709b0 0x7f4af0152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:18.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4af6398700 1 -- 172.21.15.3:0/1557170335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae00094b0 con 0x7f4af006a780 2023-03-03T06:27:18.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4af6398700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af006a780 0x7f4af00c8630 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00c590 tx=0x7f4aec00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:18.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/1557170335 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec005d90 con 0x7f4af006a780 2023-03-03T06:27:18.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/1557170335 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aec0070d0 con 0x7f4af006a780 2023-03-03T06:27:18.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/1557170335 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec007460 con 0x7f4af006a780 2023-03-03T06:27:18.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4af739a700 1 -- 172.21.15.3:0/1557170335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af015c4f0 con 0x7f4af006a780 2023-03-03T06:27:18.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4af739a700 1 -- 172.21.15.3:0/1557170335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af00cbf60 con 0x7f4af006a780 2023-03-03T06:27:18.322 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.314+0000 7f4af739a700 1 -- 172.21.15.3:0/1557170335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af015c7d0 con 0x7f4af006a780 2023-03-03T06:27:18.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.318+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/1557170335 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f4aec019030 con 0x7f4af006a780 2023-03-03T06:27:18.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.322+0000 7f4ae6ffd700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4ad4065000 0x7f4ad40674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:18.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.322+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/1557170335 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f4aec0817c0 con 0x7f4af006a780 2023-03-03T06:27:18.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.322+0000 7f4af5b97700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4ad4065000 0x7f4ad40674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:18.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.322+0000 7f4af5b97700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4ad4065000 0x7f4ad40674b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00090b0 tx=0x7f4ae001c040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:18.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.326+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/1557170335 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4aec04f580 con 0x7f4af006a780 2023-03-03T06:27:18.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.522+0000 7f4af739a700 1 -- 172.21.15.3:0/1557170335 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4af00b2640 con 0x7f4ad4065000 2023-03-03T06:27:18.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.650+0000 7f4ae6ffd700 1 -- 172.21.15.3:0/1557170335 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f4ae0011b80 con 0x7f4ad4065000 2023-03-03T06:27:18.655 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path /snap_test_dir1/dir5 2023-03-03T06:27:18.659 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.654+0000 7f4af739a700 1 -- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4ad4065000 msgr2=0x7f4ad40674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.654+0000 7f4af739a700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4ad4065000 0x7f4ad40674b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00090b0 tx=0x7f4ae001c040 comp rx=0 tx=0).stop 2023-03-03T06:27:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.654+0000 7f4af739a700 1 -- 172.21.15.3:0/1557170335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af006a780 msgr2=0x7f4af00c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.654+0000 7f4af739a700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af006a780 0x7f4af00c8630 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00c590 tx=0x7f4aec00c950 comp rx=0 tx=0).stop 2023-03-03T06:27:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.654+0000 7f4af739a700 1 -- 172.21.15.3:0/1557170335 shutdown_connections 2023-03-03T06:27:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.654+0000 7f4af739a700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4ad4065000 0x7f4ad40674b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:18.660 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.654+0000 7f4af739a700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4af00cb9a0 0x7f4af00b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.654+0000 7f4af739a700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af00709b0 0x7f4af0152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.654+0000 7f4af739a700 1 --2- 172.21.15.3:0/1557170335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4af006a780 0x7f4af00c8630 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.654+0000 7f4af739a700 1 -- 172.21.15.3:0/1557170335 >> 172.21.15.3:0/1557170335 conn(0x7f4af00607c0 msgr2=0x7f4af0156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:18.661 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.654+0000 7f4af739a700 1 -- 172.21.15.3:0/1557170335 shutdown_connections 2023-03-03T06:27:18.662 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.654+0000 7f4af739a700 1 -- 172.21.15.3:0/1557170335 wait complete. 2023-03-03T06:27:18.696 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-03-03T06:27:18.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:18.697 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-03-03T06:27:18.727 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir6 1M 2023-03-03T06:27:18.996 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.990+0000 7fd0eb08c700 1 Processor -- start 2023-03-03T06:27:18.997 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.990+0000 7fd0eb08c700 1 -- start start 2023-03-03T06:27:18.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.994+0000 7fd0eb08c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0e40b07c0 0x7fd0e40b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:18.999 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.994+0000 7fd0eb08c700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd0e40b1850 0x7fd0e40b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:19.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.994+0000 7fd0eb08c700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e406d750 con 0x7fd0e40caab0 2023-03-03T06:27:19.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.994+0000 7fd0eb08c700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e40c9510 con 0x7fd0e40b1850 2023-03-03T06:27:19.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.994+0000 7fd0eb08c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e415a640 con 0x7fd0e40b07c0 2023-03-03T06:27:19.000 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.994+0000 7fd0ea88b700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd0e40b1850 0x7fd0e40b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:19.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.994+0000 7fd0e9889700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd0e40caab0 0x7fd0e40c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42676/0 (socket says 172.21.15.3:42676) 2023-03-03T06:27:19.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.994+0000 7fd0e9889700 1 -- 172.21.15.3:0/3191952296 learned_addr learned my addr 172.21.15.3:0/3191952296 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:19.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.994+0000 7fd0ea08a700 1 --2- 172.21.15.3:0/3191952296 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0e40b07c0 0x7fd0e40b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:19.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.994+0000 7fd0ea88b700 1 -- 172.21.15.3:0/3191952296 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0e40b07c0 msgr2=0x7fd0e40b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:19.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.994+0000 7fd0ea88b700 1 --2- 172.21.15.3:0/3191952296 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0e40b07c0 0x7fd0e40b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.994+0000 7fd0ea88b700 1 -- 172.21.15.3:0/3191952296 >> v1:172.21.15.3:6789/0 conn(0x7fd0e40caab0 legacy=0x7fd0e40c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:19.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.994+0000 7fd0ea88b700 1 -- 172.21.15.3:0/3191952296 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e40b10d0 con 0x7fd0e40b1850 2023-03-03T06:27:19.002 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.998+0000 7fd0ea88b700 1 --2- 172.21.15.3:0/3191952296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e40b1850 0x7fd0e40b1cd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009d90 tx=0x7fd0dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef3936a56dbf3f0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.998+0000 7fd0e9088700 1 -- 172.21.15.3:0/3191952296 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc00e070 con 0x7fd0e40b1850 2023-03-03T06:27:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.998+0000 7fd0e9088700 1 -- 172.21.15.3:0/3191952296 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc00bb20 con 0x7fd0e40b1850 2023-03-03T06:27:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.998+0000 7fd0e9088700 1 -- 172.21.15.3:0/3191952296 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc012430 con 0x7fd0e40b1850 2023-03-03T06:27:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.998+0000 7fd0eb08c700 1 -- 172.21.15.3:0/3191952296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e40b1850 msgr2=0x7fd0e40b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:19.003 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.998+0000 7fd0eb08c700 1 --2- 172.21.15.3:0/3191952296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e40b1850 0x7fd0e40b1cd0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009d90 tx=0x7fd0dc00b330 comp rx=0 tx=0).stop 2023-03-03T06:27:19.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.998+0000 7fd0eb08c700 1 -- 172.21.15.3:0/3191952296 shutdown_connections 2023-03-03T06:27:19.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.998+0000 7fd0eb08c700 1 --2- 172.21.15.3:0/3191952296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e40b1850 0x7fd0e40b1cd0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.005 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.998+0000 7fd0eb08c700 1 --2- 172.21.15.3:0/3191952296 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0e40b07c0 0x7fd0e40b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.998+0000 7fd0eb08c700 1 -- 172.21.15.3:0/3191952296 >> 172.21.15.3:0/3191952296 conn(0x7fd0e40607c0 msgr2=0x7fd0e4060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:19.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.998+0000 7fd0eb08c700 1 -- 172.21.15.3:0/3191952296 shutdown_connections 2023-03-03T06:27:19.006 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:18.998+0000 7fd0eb08c700 1 -- 172.21.15.3:0/3191952296 wait complete. 2023-03-03T06:27:19.007 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.002+0000 7fd0eb08c700 1 Processor -- start 2023-03-03T06:27:19.008 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.002+0000 7fd0eb08c700 1 -- start start 2023-03-03T06:27:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.002+0000 7fd0eb08c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e40b07c0 0x7fd0e40b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.002+0000 7fd0eb08c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e40b1850 0x7fd0e40ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0eb08c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0e40caab0 0x7fd0e41f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0eb08c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e40cd2d0 con 0x7fd0e40b1850 2023-03-03T06:27:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0eb08c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e40ce2e0 con 0x7fd0e40b07c0 2023-03-03T06:27:19.010 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0eb08c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e41f2bc0 con 0x7fd0e40caab0 2023-03-03T06:27:19.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0e9889700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e40b1850 0x7fd0e40ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:19.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0e9889700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e40b1850 0x7fd0e40ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48998/0 (socket says 172.21.15.3:48998) 2023-03-03T06:27:19.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0e9889700 1 -- 172.21.15.3:0/2322811076 learned_addr learned my addr 172.21.15.3:0/2322811076 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:19.011 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0ea08a700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e40b07c0 0x7fd0e40b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:19.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0ea88b700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0e40caab0 0x7fd0e41f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:19.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0e9889700 1 -- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0e40caab0 msgr2=0x7fd0e41f1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:19.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0e9889700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0e40caab0 0x7fd0e41f1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.012 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0e9889700 1 -- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e40b07c0 msgr2=0x7fd0e40b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:19.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0e9889700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e40b07c0 0x7fd0e40b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0e9889700 1 -- 172.21.15.3:0/2322811076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc009a70 con 0x7fd0e40b1850 2023-03-03T06:27:19.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0ea08a700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e40b07c0 0x7fd0e40b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:27:19.013 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0e9889700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e40b1850 0x7fd0e40ccd90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400c730 tx=0x7fd0d400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0daffd700 1 -- 172.21.15.3:0/2322811076 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4005d90 con 0x7fd0e40b1850 2023-03-03T06:27:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0daffd700 1 -- 172.21.15.3:0/2322811076 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d40070d0 con 0x7fd0e40b1850 2023-03-03T06:27:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0eb08c700 1 -- 172.21.15.3:0/2322811076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e406ce20 con 0x7fd0e40b1850 2023-03-03T06:27:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0daffd700 1 -- 172.21.15.3:0/2322811076 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4007440 con 0x7fd0e40b1850 2023-03-03T06:27:19.014 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.006+0000 7fd0eb08c700 1 -- 172.21.15.3:0/2322811076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e406cb40 con 0x7fd0e40b1850 2023-03-03T06:27:19.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.010+0000 7fd0eb08c700 1 -- 172.21.15.3:0/2322811076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e406d100 con 0x7fd0e40b1850 2023-03-03T06:27:19.017 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.010+0000 7fd0daffd700 1 -- 172.21.15.3:0/2322811076 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7fd0d4019030 con 0x7fd0e40b1850 2023-03-03T06:27:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.014+0000 7fd0daffd700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd0c8065000 0x7fd0c80674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:19.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.014+0000 7fd0daffd700 1 -- 172.21.15.3:0/2322811076 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fd0d40814b0 con 0x7fd0e40b1850 2023-03-03T06:27:19.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.014+0000 7fd0ea08a700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd0c8065000 0x7fd0c80674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:19.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.018+0000 7fd0ea08a700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd0c8065000 0x7fd0c80674b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd0e41e4470 tx=0x7fd0e0009240 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:19.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.022+0000 7fd0daffd700 1 -- 172.21.15.3:0/2322811076 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd0d404a400 con 0x7fd0e40b1850 2023-03-03T06:27:19.229 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.222+0000 7fd0eb08c700 1 -- 172.21.15.3:0/2322811076 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0e40b0da0 con 0x7fd0c8065000 2023-03-03T06:27:19.358 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.354+0000 7fd0daffd700 1 -- 172.21.15.3:0/2322811076 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fd0e40b0da0 con 0x7fd0c8065000 2023-03-03T06:27:19.358 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path /snap_test_dir1/dir6 2023-03-03T06:27:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.358+0000 7fd0eb08c700 1 -- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd0c8065000 msgr2=0x7fd0c80674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.358+0000 7fd0eb08c700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd0c8065000 0x7fd0c80674b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd0e41e4470 tx=0x7fd0e0009240 comp rx=0 tx=0).stop 2023-03-03T06:27:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.358+0000 7fd0eb08c700 1 -- 172.21.15.3:0/2322811076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e40b1850 msgr2=0x7fd0e40ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.358+0000 7fd0eb08c700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e40b1850 0x7fd0e40ccd90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400c730 tx=0x7fd0d400ca40 comp rx=0 tx=0).stop 2023-03-03T06:27:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.358+0000 7fd0eb08c700 1 -- 172.21.15.3:0/2322811076 shutdown_connections 2023-03-03T06:27:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.358+0000 7fd0eb08c700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd0c8065000 0x7fd0c80674b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.358+0000 7fd0eb08c700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd0e40caab0 0x7fd0e41f1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.358+0000 7fd0eb08c700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd0e40b1850 0x7fd0e40ccd90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.358+0000 7fd0eb08c700 1 --2- 172.21.15.3:0/2322811076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0e40b07c0 0x7fd0e40b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.358+0000 7fd0eb08c700 1 -- 172.21.15.3:0/2322811076 >> 172.21.15.3:0/2322811076 conn(0x7fd0e40607c0 msgr2=0x7fd0e4159a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:19.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.358+0000 7fd0eb08c700 1 -- 172.21.15.3:0/2322811076 shutdown_connections 2023-03-03T06:27:19.365 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.358+0000 7fd0eb08c700 1 -- 172.21.15.3:0/2322811076 wait complete. 2023-03-03T06:27:19.400 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-03-03T06:27:19.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:19.401 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-03-03T06:27:19.432 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir7 1M 2023-03-03T06:27:19.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.686+0000 7f2afd710700 1 Processor -- start 2023-03-03T06:27:19.693 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.686+0000 7f2afd710700 1 -- start start 2023-03-03T06:27:19.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.690+0000 7f2afd710700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af8156ba0 0x7f2af8071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:19.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.690+0000 7f2afd710700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2af815d6e0 0x7f2af80cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:19.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.690+0000 7f2afd710700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af80cf3b0 con 0x7f2af80cfd80 2023-03-03T06:27:19.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.690+0000 7f2afd710700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af8156f70 con 0x7f2af815d6e0 2023-03-03T06:27:19.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.690+0000 7f2afd710700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80cd320 con 0x7f2af8156ba0 2023-03-03T06:27:19.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2af7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af8156ba0 0x7f2af8071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:19.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2af7fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af8156ba0 0x7f2af8071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:40226/0 (socket says 172.21.15.3:40226) 2023-03-03T06:27:19.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2af7fff700 1 -- 172.21.15.3:0/4008175369 learned_addr learned my addr 172.21.15.3:0/4008175369 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:19.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2af77fe700 1 --2- 172.21.15.3:0/4008175369 >> v2:172.21.15.32:3300/0 conn(0x7f2af815d6e0 0x7f2af80cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:19.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2af7fff700 1 -- 172.21.15.3:0/4008175369 >> v2:172.21.15.32:3300/0 conn(0x7f2af815d6e0 msgr2=0x7f2af80cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:19.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2af7fff700 1 --2- 172.21.15.3:0/4008175369 >> v2:172.21.15.32:3300/0 conn(0x7f2af815d6e0 0x7f2af80cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2af7fff700 1 -- 172.21.15.3:0/4008175369 >> v1:172.21.15.32:6789/0 conn(0x7f2af80cfd80 legacy=0x7f2af80d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:19.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2af7fff700 1 -- 172.21.15.3:0/4008175369 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80cd600 con 0x7f2af8156ba0 2023-03-03T06:27:19.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2af7fff700 1 --2- 172.21.15.3:0/4008175369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af8156ba0 0x7f2af8071280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009910 tx=0x7f2ae8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fedc859c8424a02 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:19.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2af6ffd700 1 -- 172.21.15.3:0/4008175369 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8011830 con 0x7f2af8156ba0 2023-03-03T06:27:19.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2af6ffd700 1 -- 172.21.15.3:0/4008175369 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae8011990 con 0x7f2af8156ba0 2023-03-03T06:27:19.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2af6ffd700 1 -- 172.21.15.3:0/4008175369 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8011d40 con 0x7f2af8156ba0 2023-03-03T06:27:19.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2afd710700 1 -- 172.21.15.3:0/4008175369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af8156ba0 msgr2=0x7f2af8071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:19.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2afd710700 1 --2- 172.21.15.3:0/4008175369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af8156ba0 0x7f2af8071280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009910 tx=0x7f2ae8009c20 comp rx=0 tx=0).stop 2023-03-03T06:27:19.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2afd710700 1 -- 172.21.15.3:0/4008175369 shutdown_connections 2023-03-03T06:27:19.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2afd710700 1 --2- 172.21.15.3:0/4008175369 >> v2:172.21.15.32:3300/0 conn(0x7f2af815d6e0 0x7f2af80cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2afd710700 1 --2- 172.21.15.3:0/4008175369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af8156ba0 0x7f2af8071280 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.694+0000 7f2afd710700 1 -- 172.21.15.3:0/4008175369 >> 172.21.15.3:0/4008175369 conn(0x7f2af806e5c0 msgr2=0x7f2af806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:19.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.698+0000 7f2afd710700 1 -- 172.21.15.3:0/4008175369 shutdown_connections 2023-03-03T06:27:19.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.698+0000 7f2afd710700 1 -- 172.21.15.3:0/4008175369 wait complete. 2023-03-03T06:27:19.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.698+0000 7f2afd710700 1 Processor -- start 2023-03-03T06:27:19.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.698+0000 7f2afd710700 1 -- start start 2023-03-03T06:27:19.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afd710700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af80cfd80 0x7f2af81f1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:19.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afd710700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2af8156ba0 0x7f2af80b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:19.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afd710700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af815d6e0 0x7f2af80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:19.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afd710700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80b73d0 con 0x7f2af815d6e0 2023-03-03T06:27:19.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afd710700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af8148b90 con 0x7f2af8156ba0 2023-03-03T06:27:19.706 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afd710700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80b6470 con 0x7f2af80cfd80 2023-03-03T06:27:19.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afcf0f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af815d6e0 0x7f2af80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:19.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afcf0f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af815d6e0 0x7f2af80b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49004/0 (socket says 172.21.15.3:49004) 2023-03-03T06:27:19.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afcf0f700 1 -- 172.21.15.3:0/2737103260 learned_addr learned my addr 172.21.15.3:0/2737103260 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:19.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2af77fe700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2af8156ba0 0x7f2af80b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:19.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2af7fff700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af80cfd80 0x7f2af81f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:19.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afcf0f700 1 -- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af80cfd80 msgr2=0x7f2af81f1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:19.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afcf0f700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af80cfd80 0x7f2af81f1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afcf0f700 1 -- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2af8156ba0 msgr2=0x7f2af80b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:19.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afcf0f700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2af8156ba0 0x7f2af80b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:19.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afcf0f700 1 -- 172.21.15.3:0/2737103260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae8009570 con 0x7f2af815d6e0 2023-03-03T06:27:19.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afcf0f700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af815d6e0 0x7f2af80b4330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2af000cb50 tx=0x7f2af000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:19.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2af4ff9700 1 -- 172.21.15.3:0/2737103260 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0003c20 con 0x7f2af815d6e0 2023-03-03T06:27:19.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2af4ff9700 1 -- 172.21.15.3:0/2737103260 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af0003d80 con 0x7f2af815d6e0 2023-03-03T06:27:19.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2af4ff9700 1 -- 172.21.15.3:0/2737103260 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000aac0 con 0x7f2af815d6e0 2023-03-03T06:27:19.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afd710700 1 -- 172.21.15.3:0/2737103260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af80717c0 con 0x7f2af815d6e0 2023-03-03T06:27:19.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.702+0000 7f2afd710700 1 -- 172.21.15.3:0/2737103260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af80cd8e0 con 0x7f2af815d6e0 2023-03-03T06:27:19.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.706+0000 7f2afd710700 1 -- 172.21.15.3:0/2737103260 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af80cef40 con 0x7f2af815d6e0 2023-03-03T06:27:19.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.706+0000 7f2af4ff9700 1 -- 172.21.15.3:0/2737103260 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f2af0010050 con 0x7f2af815d6e0 2023-03-03T06:27:19.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.710+0000 7f2af4ff9700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2ae4065000 0x7f2ae40674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:19.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.710+0000 7f2af4ff9700 1 -- 172.21.15.3:0/2737103260 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f2af0082160 con 0x7f2af815d6e0 2023-03-03T06:27:19.715 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.710+0000 7f2af7fff700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2ae4065000 0x7f2ae40674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:19.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.710+0000 7f2af7fff700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2ae4065000 0x7f2ae40674b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009c60 tx=0x7f2ae8008030 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:19.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.714+0000 7f2af4ff9700 1 -- 172.21.15.3:0/2737103260 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f2af004ff20 con 0x7f2af815d6e0 2023-03-03T06:27:19.916 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:19.910+0000 7f2afd710700 1 -- 172.21.15.3:0/2737103260 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2af8067880 con 0x7f2ae4065000 2023-03-03T06:27:20.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.038+0000 7f2af4ff9700 1 -- 172.21.15.3:0/2737103260 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f2af8067880 con 0x7f2ae4065000 2023-03-03T06:27:20.044 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path /snap_test_dir1/dir7 2023-03-03T06:27:20.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.042+0000 7f2afd710700 1 -- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2ae4065000 msgr2=0x7f2ae40674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:20.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.042+0000 7f2afd710700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2ae4065000 0x7f2ae40674b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009c60 tx=0x7f2ae8008030 comp rx=0 tx=0).stop 2023-03-03T06:27:20.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.042+0000 7f2afd710700 1 -- 172.21.15.3:0/2737103260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af815d6e0 msgr2=0x7f2af80b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:20.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.042+0000 7f2afd710700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af815d6e0 0x7f2af80b4330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2af000cb50 tx=0x7f2af000ce60 comp rx=0 tx=0).stop 2023-03-03T06:27:20.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.042+0000 7f2afd710700 1 -- 172.21.15.3:0/2737103260 shutdown_connections 2023-03-03T06:27:20.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.042+0000 7f2afd710700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2ae4065000 0x7f2ae40674b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:20.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.042+0000 7f2afd710700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2af815d6e0 0x7f2af80b4330 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:20.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.042+0000 7f2afd710700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2af8156ba0 0x7f2af80b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:20.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.042+0000 7f2afd710700 1 --2- 172.21.15.3:0/2737103260 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2af80cfd80 0x7f2af81f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:20.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.042+0000 7f2afd710700 1 -- 172.21.15.3:0/2737103260 >> 172.21.15.3:0/2737103260 conn(0x7f2af806e5c0 msgr2=0x7f2af815dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:20.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.042+0000 7f2afd710700 1 -- 172.21.15.3:0/2737103260 shutdown_connections 2023-03-03T06:27:20.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.042+0000 7f2afd710700 1 -- 172.21.15.3:0/2737103260 wait complete. 2023-03-03T06:27:20.082 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-03-03T06:27:20.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:20.083 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-03-03T06:27:20.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir8 1M 2023-03-03T06:27:20.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.374+0000 7f1f00fb4700 1 Processor -- start 2023-03-03T06:27:20.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.374+0000 7f1f00fb4700 1 -- start start 2023-03-03T06:27:20.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1f00fb4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1efc0b12d0 0x7f1efc0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:20.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1f00fb4700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1efc06eef0 0x7f1efc06f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:20.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1f00fb4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc0b1c60 con 0x7f1efc0b51d0 2023-03-03T06:27:20.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1f00fb4700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1efc0c9e10 con 0x7f1efc06eef0 2023-03-03T06:27:20.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1f00fb4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc06ea60 con 0x7f1efc0b12d0 2023-03-03T06:27:20.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1efbfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1efc06eef0 0x7f1efc06f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:20.383 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1efbfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1efc06eef0 0x7f1efc06f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52578/0 (socket says 172.21.15.3:52578) 2023-03-03T06:27:20.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1efbfff700 1 -- 172.21.15.3:0/4201312224 learned_addr learned my addr 172.21.15.3:0/4201312224 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:20.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1efaffd700 1 --2- 172.21.15.3:0/4201312224 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1efc0b12d0 0x7f1efc0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:20.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1efaffd700 1 -- 172.21.15.3:0/4201312224 >> v2:172.21.15.3:3300/0 conn(0x7f1efc06eef0 msgr2=0x7f1efc06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:20.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1efaffd700 1 --2- 172.21.15.3:0/4201312224 >> v2:172.21.15.3:3300/0 conn(0x7f1efc06eef0 0x7f1efc06f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:20.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1efaffd700 1 -- 172.21.15.3:0/4201312224 >> v1:172.21.15.32:6789/0 conn(0x7f1efc0b51d0 legacy=0x7f1efc0cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:20.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1efaffd700 1 -- 172.21.15.3:0/4201312224 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc15a850 con 0x7f1efc0b12d0 2023-03-03T06:27:20.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.378+0000 7f1efbfff700 1 --2- 172.21.15.3:0/4201312224 >> v2:172.21.15.3:3300/0 conn(0x7f1efc06eef0 0x7f1efc06f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:27:20.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.382+0000 7f1efaffd700 1 --2- 172.21.15.3:0/4201312224 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1efc0b12d0 0x7f1efc0b1720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009850 tx=0x7f1ef0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23bc051c8af7b7b1 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:20.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.382+0000 7f1efa7fc700 1 -- 172.21.15.3:0/4201312224 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0011830 con 0x7f1efc0b12d0 2023-03-03T06:27:20.386 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.382+0000 7f1efa7fc700 1 -- 172.21.15.3:0/4201312224 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef0011990 con 0x7f1efc0b12d0 2023-03-03T06:27:20.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.382+0000 7f1efa7fc700 1 -- 172.21.15.3:0/4201312224 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0011d40 con 0x7f1efc0b12d0 2023-03-03T06:27:20.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.382+0000 7f1f00fb4700 1 -- 172.21.15.3:0/4201312224 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1efc0b12d0 msgr2=0x7f1efc0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:20.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.382+0000 7f1f00fb4700 1 --2- 172.21.15.3:0/4201312224 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1efc0b12d0 0x7f1efc0b1720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009850 tx=0x7f1ef0009b60 comp rx=0 tx=0).stop 2023-03-03T06:27:20.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.382+0000 7f1f00fb4700 1 -- 172.21.15.3:0/4201312224 shutdown_connections 2023-03-03T06:27:20.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.382+0000 7f1f00fb4700 1 --2- 172.21.15.3:0/4201312224 >> v2:172.21.15.3:3300/0 conn(0x7f1efc06eef0 0x7f1efc06f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:20.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.382+0000 7f1f00fb4700 1 --2- 172.21.15.3:0/4201312224 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1efc0b12d0 0x7f1efc0b1720 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:20.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.382+0000 7f1f00fb4700 1 -- 172.21.15.3:0/4201312224 >> 172.21.15.3:0/4201312224 conn(0x7f1efc069d70 msgr2=0x7f1efc06a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:20.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.382+0000 7f1f00fb4700 1 -- 172.21.15.3:0/4201312224 shutdown_connections 2023-03-03T06:27:20.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.382+0000 7f1f00fb4700 1 -- 172.21.15.3:0/4201312224 wait complete. 2023-03-03T06:27:20.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.386+0000 7f1f00fb4700 1 Processor -- start 2023-03-03T06:27:20.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.386+0000 7f1f00fb4700 1 -- start start 2023-03-03T06:27:20.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.386+0000 7f1f00fb4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1efc06eef0 0x7f1efc149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:20.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.386+0000 7f1f00fb4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1efc0b12d0 0x7f1efc0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:20.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.386+0000 7f1f00fb4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1efc0b51d0 0x7f1efc0af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:20.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.386+0000 7f1f00fb4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0c7d30 con 0x7f1efc0b12d0 2023-03-03T06:27:20.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.386+0000 7f1f00fb4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc14a260 con 0x7f1efc06eef0 2023-03-03T06:27:20.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.386+0000 7f1f00fb4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0c8d80 con 0x7f1efc0b51d0 2023-03-03T06:27:20.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.386+0000 7f1efaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1efc0b12d0 0x7f1efc0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.386+0000 7f1efb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1efc06eef0 0x7f1efc149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.386+0000 7f1efaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1efc0b12d0 0x7f1efc0c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52594/0 (socket says 172.21.15.3:52594) 2023-03-03T06:27:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.386+0000 7f1efbfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1efc0b51d0 0x7f1efc0af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.386+0000 7f1efaffd700 1 -- 172.21.15.3:0/766836384 learned_addr learned my addr 172.21.15.3:0/766836384 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.390+0000 7f1efaffd700 1 -- 172.21.15.3:0/766836384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1efc0b51d0 msgr2=0x7f1efc0af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:20.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.390+0000 7f1efaffd700 1 --2- 172.21.15.3:0/766836384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1efc0b51d0 0x7f1efc0af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.390+0000 7f1efaffd700 1 -- 172.21.15.3:0/766836384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1efc06eef0 msgr2=0x7f1efc149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.390+0000 7f1efaffd700 1 --2- 172.21.15.3:0/766836384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1efc06eef0 0x7f1efc149d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.390+0000 7f1efaffd700 1 -- 172.21.15.3:0/766836384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef4009b30 con 0x7f1efc0b12d0 2023-03-03T06:27:20.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.390+0000 7f1efaffd700 1 --2- 172.21.15.3:0/766836384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1efc0b12d0 0x7f1efc0c8840 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0014040 tx=0x7f1ef0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:20.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.390+0000 7f1ee3fff700 1 -- 172.21.15.3:0/766836384 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00036c0 con 0x7f1efc0b12d0 2023-03-03T06:27:20.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.390+0000 7f1ee3fff700 1 -- 172.21.15.3:0/766836384 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef001e070 con 0x7f1efc0b12d0 2023-03-03T06:27:20.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.390+0000 7f1ee3fff700 1 -- 172.21.15.3:0/766836384 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0003a80 con 0x7f1efc0b12d0 2023-03-03T06:27:20.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.390+0000 7f1f00fb4700 1 -- 172.21.15.3:0/766836384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef00094b0 con 0x7f1efc0b12d0 2023-03-03T06:27:20.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.390+0000 7f1f00fb4700 1 -- 172.21.15.3:0/766836384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc0ca340 con 0x7f1efc0b12d0 2023-03-03T06:27:20.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.390+0000 7f1f00fb4700 1 -- 172.21.15.3:0/766836384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1efc0ca060 con 0x7f1efc0b12d0 2023-03-03T06:27:20.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.394+0000 7f1ee3fff700 1 -- 172.21.15.3:0/766836384 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f1ef000f050 con 0x7f1efc0b12d0 2023-03-03T06:27:20.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.394+0000 7f1ee3fff700 1 --2- 172.21.15.3:0/766836384 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1ee8065000 0x7f1ee80674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:20.401 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.394+0000 7f1ee3fff700 1 -- 172.21.15.3:0/766836384 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f1ef0020070 con 0x7f1efc0b12d0 2023-03-03T06:27:20.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.398+0000 7f1efb7fe700 1 --2- 172.21.15.3:0/766836384 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1ee8065000 0x7f1ee80674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:20.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.398+0000 7f1efb7fe700 1 --2- 172.21.15.3:0/766836384 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1ee8065000 0x7f1ee80674b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1efc15aa90 tx=0x7f1eec009040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:20.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.402+0000 7f1ee3fff700 1 -- 172.21.15.3:0/766836384 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1ef0051500 con 0x7f1efc0b12d0 2023-03-03T06:27:20.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.598+0000 7f1f00fb4700 1 -- 172.21.15.3:0/766836384 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1efc06e6f0 con 0x7f1ee8065000 2023-03-03T06:27:20.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.722+0000 7f1ee3fff700 1 -- 172.21.15.3:0/766836384 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1efc06e6f0 con 0x7f1ee8065000 2023-03-03T06:27:20.728 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path /snap_test_dir1/dir8 2023-03-03T06:27:20.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.726+0000 7f1f00fb4700 1 -- 172.21.15.3:0/766836384 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1ee8065000 msgr2=0x7f1ee80674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:20.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.726+0000 7f1f00fb4700 1 --2- 172.21.15.3:0/766836384 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1ee8065000 0x7f1ee80674b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1efc15aa90 tx=0x7f1eec009040 comp rx=0 tx=0).stop 2023-03-03T06:27:20.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.726+0000 7f1f00fb4700 1 -- 172.21.15.3:0/766836384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1efc0b12d0 msgr2=0x7f1efc0c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:20.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.726+0000 7f1f00fb4700 1 --2- 172.21.15.3:0/766836384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1efc0b12d0 0x7f1efc0c8840 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0014040 tx=0x7f1ef0004770 comp rx=0 tx=0).stop 2023-03-03T06:27:20.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.726+0000 7f1f00fb4700 1 -- 172.21.15.3:0/766836384 shutdown_connections 2023-03-03T06:27:20.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.726+0000 7f1f00fb4700 1 --2- 172.21.15.3:0/766836384 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f1ee8065000 0x7f1ee80674b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:20.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.726+0000 7f1f00fb4700 1 --2- 172.21.15.3:0/766836384 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f1efc0b51d0 0x7f1efc0af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:20.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.726+0000 7f1f00fb4700 1 --2- 172.21.15.3:0/766836384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1efc0b12d0 0x7f1efc0c8840 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:20.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.726+0000 7f1f00fb4700 1 --2- 172.21.15.3:0/766836384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1efc06eef0 0x7f1efc149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:20.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.726+0000 7f1f00fb4700 1 -- 172.21.15.3:0/766836384 >> 172.21.15.3:0/766836384 conn(0x7f1efc069d70 msgr2=0x7f1efc0b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:20.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.730+0000 7f1f00fb4700 1 -- 172.21.15.3:0/766836384 shutdown_connections 2023-03-03T06:27:20.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:20.730+0000 7f1f00fb4700 1 -- 172.21.15.3:0/766836384 wait complete. 2023-03-03T06:27:20.771 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-03-03T06:27:20.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:27:20.772 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-03-03T06:27:20.800 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir9 1M 2023-03-03T06:27:21.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.058+0000 7f9872620700 1 Processor -- start 2023-03-03T06:27:21.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.058+0000 7f9872620700 1 -- start start 2023-03-03T06:27:21.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f9872620700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f986c0c8fe0 0x7f986c0c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:21.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f9872620700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986c06c090 con 0x7f986c0ccc70 2023-03-03T06:27:21.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f987161e700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f986c0ccc70 0x7f986c15ce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43038/0 (socket says 172.21.15.3:43038) 2023-03-03T06:27:21.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f987161e700 1 -- 172.21.15.3:0/374406273 learned_addr learned my addr 172.21.15.3:0/374406273 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:21.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f9872620700 1 -- 172.21.15.3:0/374406273 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986c0cd230 con 0x7f986c158810 2023-03-03T06:27:21.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f9871e1f700 1 --2- 172.21.15.3:0/374406273 >> v2:172.21.15.3:3300/0 conn(0x7f986c0c8fe0 0x7f986c0c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:21.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f9872620700 1 -- 172.21.15.3:0/374406273 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f986c0cd040 con 0x7f986c0c8fe0 2023-03-03T06:27:21.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f9863fff700 1 -- 172.21.15.3:0/374406273 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654888491 0 0) 0x7f986c06c090 con 0x7f986c0ccc70 2023-03-03T06:27:21.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f9863fff700 1 -- 172.21.15.3:0/374406273 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98580035d0 con 0x7f986c0ccc70 2023-03-03T06:27:21.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f9863fff700 1 -- 172.21.15.3:0/374406273 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096701644 0 0) 0x7f986c0cd230 con 0x7f986c158810 2023-03-03T06:27:21.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f9863fff700 1 -- 172.21.15.3:0/374406273 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f986c06c090 con 0x7f986c158810 2023-03-03T06:27:21.070 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f9871e1f700 1 -- 172.21.15.3:0/374406273 >> v1:172.21.15.32:6789/0 conn(0x7f986c158810 legacy=0x7f986c0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:21.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f9871e1f700 1 -- 172.21.15.3:0/374406273 >> v1:172.21.15.3:6789/0 conn(0x7f986c0ccc70 legacy=0x7f986c15ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:21.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.062+0000 7f9871e1f700 1 -- 172.21.15.3:0/374406273 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c06c2e0 con 0x7f986c0c8fe0 2023-03-03T06:27:21.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.066+0000 7f9871e1f700 1 --2- 172.21.15.3:0/374406273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f986c0c8fe0 0x7f986c0c93e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9864009d90 tx=0x7f986400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668989ac8536c2c9 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:21.071 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.066+0000 7f9863fff700 1 -- 172.21.15.3:0/374406273 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986400e070 con 0x7f986c0c8fe0 2023-03-03T06:27:21.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.066+0000 7f9863fff700 1 -- 172.21.15.3:0/374406273 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986400bb20 con 0x7f986c0c8fe0 2023-03-03T06:27:21.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.066+0000 7f9863fff700 1 -- 172.21.15.3:0/374406273 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864012430 con 0x7f986c0c8fe0 2023-03-03T06:27:21.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.066+0000 7f9872620700 1 -- 172.21.15.3:0/374406273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f986c0c8fe0 msgr2=0x7f986c0c93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:21.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.066+0000 7f9872620700 1 --2- 172.21.15.3:0/374406273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f986c0c8fe0 0x7f986c0c93e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9864009d90 tx=0x7f986400b330 comp rx=0 tx=0).stop 2023-03-03T06:27:21.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.066+0000 7f9872620700 1 -- 172.21.15.3:0/374406273 shutdown_connections 2023-03-03T06:27:21.072 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.066+0000 7f9872620700 1 --2- 172.21.15.3:0/374406273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f986c0c8fe0 0x7f986c0c93e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:21.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.066+0000 7f9872620700 1 -- 172.21.15.3:0/374406273 >> 172.21.15.3:0/374406273 conn(0x7f986c0607c0 msgr2=0x7f986c060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:21.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.066+0000 7f9872620700 1 -- 172.21.15.3:0/374406273 shutdown_connections 2023-03-03T06:27:21.073 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.066+0000 7f9872620700 1 -- 172.21.15.3:0/374406273 wait complete. 2023-03-03T06:27:21.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.070+0000 7f9872620700 1 Processor -- start 2023-03-03T06:27:21.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.070+0000 7f9872620700 1 -- start start 2023-03-03T06:27:21.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.070+0000 7f9872620700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f986c0c8fe0 0x7f986c1f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:21.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.070+0000 7f9872620700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f986c0ccc70 0x7f986c155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:21.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.070+0000 7f9872620700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f986c158810 0x7f986c06ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:21.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.070+0000 7f9872620700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c0b16a0 con 0x7f986c158810 2023-03-03T06:27:21.077 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.070+0000 7f9872620700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c1f5d50 con 0x7f986c0ccc70 2023-03-03T06:27:21.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.070+0000 7f9872620700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986c155db0 con 0x7f986c0c8fe0 2023-03-03T06:27:21.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9871e1f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f986c158810 0x7f986c06ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:21.078 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f987161e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f986c0c8fe0 0x7f986c1f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:21.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9871e1f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f986c158810 0x7f986c06ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52614/0 (socket says 172.21.15.3:52614) 2023-03-03T06:27:21.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9871e1f700 1 -- 172.21.15.3:0/1640634483 learned_addr learned my addr 172.21.15.3:0/1640634483 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:21.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9870e1d700 1 --2- 172.21.15.3:0/1640634483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f986c0ccc70 0x7f986c155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:21.079 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9871e1f700 1 -- 172.21.15.3:0/1640634483 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f986c0c8fe0 msgr2=0x7f986c1f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:21.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9871e1f700 1 --2- 172.21.15.3:0/1640634483 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f986c0c8fe0 0x7f986c1f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:21.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9871e1f700 1 -- 172.21.15.3:0/1640634483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f986c0ccc70 msgr2=0x7f986c155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:21.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9871e1f700 1 --2- 172.21.15.3:0/1640634483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f986c0ccc70 0x7f986c155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:21.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9871e1f700 1 -- 172.21.15.3:0/1640634483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864009a70 con 0x7f986c158810 2023-03-03T06:27:21.080 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9871e1f700 1 --2- 172.21.15.3:0/1640634483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f986c158810 0x7f986c06ccc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f986400b680 tx=0x7f9864004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:21.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9861ffb700 1 -- 172.21.15.3:0/1640634483 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986401c810 con 0x7f986c158810 2023-03-03T06:27:21.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9861ffb700 1 -- 172.21.15.3:0/1640634483 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9864020070 con 0x7f986c158810 2023-03-03T06:27:21.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9861ffb700 1 -- 172.21.15.3:0/1640634483 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986401cb90 con 0x7f986c158810 2023-03-03T06:27:21.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9872620700 1 -- 172.21.15.3:0/1640634483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c06c8a0 con 0x7f986c158810 2023-03-03T06:27:21.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.074+0000 7f9872620700 1 -- 172.21.15.3:0/1640634483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c06c5c0 con 0x7f986c158810 2023-03-03T06:27:21.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.078+0000 7f9872620700 1 -- 172.21.15.3:0/1640634483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986c158c70 con 0x7f986c158810 2023-03-03T06:27:21.085 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.078+0000 7f9861ffb700 1 -- 172.21.15.3:0/1640634483 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f986402e070 con 0x7f986c158810 2023-03-03T06:27:21.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.082+0000 7f9861ffb700 1 --2- 172.21.15.3:0/1640634483 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9850065000 0x7f98500674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:21.087 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.082+0000 7f9861ffb700 1 -- 172.21.15.3:0/1640634483 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f9864022070 con 0x7f986c158810 2023-03-03T06:27:21.088 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.082+0000 7f987161e700 1 --2- 172.21.15.3:0/1640634483 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9850065000 0x7f98500674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:21.089 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.082+0000 7f987161e700 1 --2- 172.21.15.3:0/1640634483 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9850065000 0x7f98500674b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9868000f60 tx=0x7f9868007e20 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:21.093 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.086+0000 7f9861ffb700 1 -- 172.21.15.3:0/1640634483 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f98640b4400 con 0x7f986c158810 2023-03-03T06:27:21.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.286+0000 7f9872620700 1 -- 172.21.15.3:0/1640634483 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f986c0b17e0 con 0x7f9850065000 2023-03-03T06:27:21.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.410+0000 7f9861ffb700 1 -- 172.21.15.3:0/1640634483 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f986c0b17e0 con 0x7f9850065000 2023-03-03T06:27:21.416 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path /snap_test_dir1/dir9 2023-03-03T06:27:21.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.414+0000 7f9872620700 1 -- 172.21.15.3:0/1640634483 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9850065000 msgr2=0x7f98500674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:21.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.414+0000 7f9872620700 1 --2- 172.21.15.3:0/1640634483 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9850065000 0x7f98500674b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9868000f60 tx=0x7f9868007e20 comp rx=0 tx=0).stop 2023-03-03T06:27:21.422 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.414+0000 7f9872620700 1 -- 172.21.15.3:0/1640634483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f986c158810 msgr2=0x7f986c06ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:21.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.414+0000 7f9872620700 1 --2- 172.21.15.3:0/1640634483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f986c158810 0x7f986c06ccc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f986400b680 tx=0x7f9864004af0 comp rx=0 tx=0).stop 2023-03-03T06:27:21.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.418+0000 7f9872620700 1 -- 172.21.15.3:0/1640634483 shutdown_connections 2023-03-03T06:27:21.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.418+0000 7f9872620700 1 --2- 172.21.15.3:0/1640634483 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9850065000 0x7f98500674b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:21.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.418+0000 7f9872620700 1 --2- 172.21.15.3:0/1640634483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f986c158810 0x7f986c06ccc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:21.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.418+0000 7f9872620700 1 --2- 172.21.15.3:0/1640634483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f986c0ccc70 0x7f986c155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:21.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.418+0000 7f9872620700 1 --2- 172.21.15.3:0/1640634483 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f986c0c8fe0 0x7f986c1f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:21.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.418+0000 7f9872620700 1 -- 172.21.15.3:0/1640634483 >> 172.21.15.3:0/1640634483 conn(0x7f986c0607c0 msgr2=0x7f986c0c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:21.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.418+0000 7f9872620700 1 -- 172.21.15.3:0/1640634483 shutdown_connections 2023-03-03T06:27:21.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.418+0000 7f9872620700 1 -- 172.21.15.3:0/1640634483 wait complete. 2023-03-03T06:27:21.456 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-03-03T06:27:21.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir0 1M 2023-03-03T06:27:21.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.666+0000 7f94a4884700 1 Processor -- start 2023-03-03T06:27:21.672 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.666+0000 7f94a4884700 1 -- start start 2023-03-03T06:27:21.675 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.670+0000 7f94a4884700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a015d6e0 0x7f94a00cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:21.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.670+0000 7f94a4884700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a00cf3b0 con 0x7f94a0156ba0 2023-03-03T06:27:21.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.670+0000 7f94a4884700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a0156f70 con 0x7f94a00cfd80 2023-03-03T06:27:21.676 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.670+0000 7f94a4884700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00cd320 con 0x7f94a015d6e0 2023-03-03T06:27:21.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.670+0000 7f949f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f94a0156ba0 0x7f94a0071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43052/0 (socket says 172.21.15.3:43052) 2023-03-03T06:27:21.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.670+0000 7f949f7fe700 1 -- 172.21.15.3:0/1732293548 learned_addr learned my addr 172.21.15.3:0/1732293548 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:21.677 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.670+0000 7f949effd700 1 --2- 172.21.15.3:0/1732293548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a015d6e0 0x7f94a00cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:21.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f949e7fc700 1 -- 172.21.15.3:0/1732293548 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005847848 0 0) 0x7f94a0156f70 con 0x7f94a00cfd80 2023-03-03T06:27:21.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f949e7fc700 1 -- 172.21.15.3:0/1732293548 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94880035d0 con 0x7f94a00cfd80 2023-03-03T06:27:21.678 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f949e7fc700 1 -- 172.21.15.3:0/1732293548 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609619539 0 0) 0x7f94a00cf3b0 con 0x7f94a0156ba0 2023-03-03T06:27:21.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f949e7fc700 1 -- 172.21.15.3:0/1732293548 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a0156f70 con 0x7f94a0156ba0 2023-03-03T06:27:21.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f949effd700 1 -- 172.21.15.3:0/1732293548 >> v1:172.21.15.32:6789/0 conn(0x7f94a00cfd80 legacy=0x7f94a00d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:21.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f949effd700 1 -- 172.21.15.3:0/1732293548 >> v1:172.21.15.3:6789/0 conn(0x7f94a0156ba0 legacy=0x7f94a0071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:21.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f949effd700 1 -- 172.21.15.3:0/1732293548 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a00cd600 con 0x7f94a015d6e0 2023-03-03T06:27:21.679 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f949effd700 1 --2- 172.21.15.3:0/1732293548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a015d6e0 0x7f94a00cf840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9494009880 tx=0x7f9494009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=693f73b1e48d3ba8 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:21.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f949e7fc700 1 -- 172.21.15.3:0/1732293548 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494011830 con 0x7f94a015d6e0 2023-03-03T06:27:21.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f949e7fc700 1 -- 172.21.15.3:0/1732293548 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9494011990 con 0x7f94a015d6e0 2023-03-03T06:27:21.680 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f949e7fc700 1 -- 172.21.15.3:0/1732293548 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494011d20 con 0x7f94a015d6e0 2023-03-03T06:27:21.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f94a4884700 1 -- 172.21.15.3:0/1732293548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a015d6e0 msgr2=0x7f94a00cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:21.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f94a4884700 1 --2- 172.21.15.3:0/1732293548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a015d6e0 0x7f94a00cf840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9494009880 tx=0x7f9494009b90 comp rx=0 tx=0).stop 2023-03-03T06:27:21.681 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f94a4884700 1 -- 172.21.15.3:0/1732293548 shutdown_connections 2023-03-03T06:27:21.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f94a4884700 1 --2- 172.21.15.3:0/1732293548 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a015d6e0 0x7f94a00cf840 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:21.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f94a4884700 1 -- 172.21.15.3:0/1732293548 >> 172.21.15.3:0/1732293548 conn(0x7f94a006e5c0 msgr2=0x7f94a006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:21.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f94a4884700 1 -- 172.21.15.3:0/1732293548 shutdown_connections 2023-03-03T06:27:21.682 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.674+0000 7f94a4884700 1 -- 172.21.15.3:0/1732293548 wait complete. 2023-03-03T06:27:21.683 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.678+0000 7f94a4884700 1 Processor -- start 2023-03-03T06:27:21.684 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.678+0000 7f94a4884700 1 -- start start 2023-03-03T06:27:21.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.678+0000 7f94a4884700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a00cfd80 0x7f94a00c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:21.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.678+0000 7f94a4884700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a0156ba0 0x7f94a0157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:21.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.678+0000 7f94a4884700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a015d6e0 0x7f94a0157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:21.685 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.678+0000 7f94a4884700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00b2670 con 0x7f94a015d6e0 2023-03-03T06:27:21.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.678+0000 7f94a4884700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0158d20 con 0x7f94a0156ba0 2023-03-03T06:27:21.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.678+0000 7f94a4884700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0159000 con 0x7f94a00cfd80 2023-03-03T06:27:21.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f949f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a00cfd80 0x7f94a00c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:21.686 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f949effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a0156ba0 0x7f94a0157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:21.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f949f7fe700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a00cfd80 0x7f94a00c5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:49978/0 (socket says 172.21.15.3:49978) 2023-03-03T06:27:21.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f949ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a015d6e0 0x7f94a0157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:21.687 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f949f7fe700 1 -- 172.21.15.3:0/346300463 learned_addr learned my addr 172.21.15.3:0/346300463 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f949f7fe700 1 -- 172.21.15.3:0/346300463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a0156ba0 msgr2=0x7f94a0157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f949f7fe700 1 --2- 172.21.15.3:0/346300463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a0156ba0 0x7f94a0157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f949f7fe700 1 -- 172.21.15.3:0/346300463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a015d6e0 msgr2=0x7f94a0157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f949f7fe700 1 --2- 172.21.15.3:0/346300463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a015d6e0 0x7f94a0157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f949f7fe700 1 -- 172.21.15.3:0/346300463 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94940094b0 con 0x7f94a00cfd80 2023-03-03T06:27:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f949f7fe700 1 --2- 172.21.15.3:0/346300463 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a00cfd80 0x7f94a00c5a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f949000c730 tx=0x7f949000ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f9487fff700 1 -- 172.21.15.3:0/346300463 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490005d90 con 0x7f94a00cfd80 2023-03-03T06:27:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f9487fff700 1 -- 172.21.15.3:0/346300463 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94900070d0 con 0x7f94a00cfd80 2023-03-03T06:27:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f9487fff700 1 -- 172.21.15.3:0/346300463 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490007460 con 0x7f94a00cfd80 2023-03-03T06:27:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f94a4884700 1 -- 172.21.15.3:0/346300463 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a00cef40 con 0x7f94a00cfd80 2023-03-03T06:27:21.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.682+0000 7f94a4884700 1 -- 172.21.15.3:0/346300463 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a00cd8e0 con 0x7f94a00cfd80 2023-03-03T06:27:21.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.686+0000 7f94a4884700 1 -- 172.21.15.3:0/346300463 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a00cf0d0 con 0x7f94a00cfd80 2023-03-03T06:27:21.692 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.686+0000 7f9487fff700 1 -- 172.21.15.3:0/346300463 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f9490019030 con 0x7f94a00cfd80 2023-03-03T06:27:21.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.690+0000 7f9487fff700 1 --2- 172.21.15.3:0/346300463 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f948c065000 0x7f948c0674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:21.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.690+0000 7f9487fff700 1 -- 172.21.15.3:0/346300463 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f9490081660 con 0x7f94a00cfd80 2023-03-03T06:27:21.694 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.690+0000 7f949effd700 1 --2- 172.21.15.3:0/346300463 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f948c065000 0x7f948c0674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:21.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.690+0000 7f949effd700 1 --2- 172.21.15.3:0/346300463 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f948c065000 0x7f948c0674b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f94940090b0 tx=0x7f949401c040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:21.704 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.698+0000 7f9487fff700 1 -- 172.21.15.3:0/346300463 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f949004f420 con 0x7f94a00cfd80 2023-03-03T06:27:21.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:21.898+0000 7f94a4884700 1 -- 172.21.15.3:0/346300463 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f94a0159520 con 0x7f948c065000 2023-03-03T06:27:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.010+0000 7f9487fff700 1 -- 172.21.15.3:0/346300463 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f94a0159520 con 0x7f948c065000 2023-03-03T06:27:22.016 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-03-03T06:27:22.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.014+0000 7f94a4884700 1 -- 172.21.15.3:0/346300463 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f948c065000 msgr2=0x7f948c0674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.014+0000 7f94a4884700 1 --2- 172.21.15.3:0/346300463 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f948c065000 0x7f948c0674b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f94940090b0 tx=0x7f949401c040 comp rx=0 tx=0).stop 2023-03-03T06:27:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.014+0000 7f94a4884700 1 -- 172.21.15.3:0/346300463 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a00cfd80 msgr2=0x7f94a00c5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.014+0000 7f94a4884700 1 --2- 172.21.15.3:0/346300463 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a00cfd80 0x7f94a00c5a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f949000c730 tx=0x7f949000ca40 comp rx=0 tx=0).stop 2023-03-03T06:27:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.014+0000 7f94a4884700 1 -- 172.21.15.3:0/346300463 shutdown_connections 2023-03-03T06:27:22.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.014+0000 7f94a4884700 1 --2- 172.21.15.3:0/346300463 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f948c065000 0x7f948c0674b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.014+0000 7f94a4884700 1 --2- 172.21.15.3:0/346300463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94a015d6e0 0x7f94a0157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.014+0000 7f94a4884700 1 --2- 172.21.15.3:0/346300463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a0156ba0 0x7f94a0157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.014+0000 7f94a4884700 1 --2- 172.21.15.3:0/346300463 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f94a00cfd80 0x7f94a00c5a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.014+0000 7f94a4884700 1 -- 172.21.15.3:0/346300463 >> 172.21.15.3:0/346300463 conn(0x7f94a006e5c0 msgr2=0x7f94a015ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:22.022 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.018+0000 7f94a4884700 1 -- 172.21.15.3:0/346300463 shutdown_connections 2023-03-03T06:27:22.023 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.018+0000 7f94a4884700 1 -- 172.21.15.3:0/346300463 wait complete. 2023-03-03T06:27:22.058 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-03-03T06:27:22.058 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir1 1M 2023-03-03T06:27:22.270 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.266+0000 7f6169bc3700 1 Processor -- start 2023-03-03T06:27:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.266+0000 7f6169bc3700 1 -- start start 2023-03-03T06:27:22.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f6169bc3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f616415d6e0 0x7f61640cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:22.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f6169bc3700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61640cf3b0 con 0x7f6164156ba0 2023-03-03T06:27:22.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f6169bc3700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6164156f70 con 0x7f61640cfd80 2023-03-03T06:27:22.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f6169bc3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61640cd320 con 0x7f616415d6e0 2023-03-03T06:27:22.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f6168bc1700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6164156ba0 0x7f6164071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43068/0 (socket says 172.21.15.3:43068) 2023-03-03T06:27:22.275 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f6168bc1700 1 -- 172.21.15.3:0/2325914612 learned_addr learned my addr 172.21.15.3:0/2325914612 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:22.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f6163fff700 1 --2- 172.21.15.3:0/2325914612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f616415d6e0 0x7f61640cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:22.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317053032 0 0) 0x7f61640cf3b0 con 0x7f6164156ba0 2023-03-03T06:27:22.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61540035d0 con 0x7f6164156ba0 2023-03-03T06:27:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3535594338 0 0) 0x7f61540035d0 con 0x7f6164156ba0 2023-03-03T06:27:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f616415d6e0 msgr2=0x7f61640cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 --2- 172.21.15.3:0/2325914612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f616415d6e0 0x7f61640cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 >> v1:172.21.15.32:6789/0 conn(0x7f61640cfd80 legacy=0x7f61640d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640cd600 con 0x7f6164156ba0 2023-03-03T06:27:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f6158001670 con 0x7f6164156ba0 2023-03-03T06:27:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 >> v1:172.21.15.3:6789/0 conn(0x7f6164156ba0 legacy=0x7f6164071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 --2- 172.21.15.3:0/2325914612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6154003a50 0x7f6154003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 --2- 172.21.15.3:0/2325914612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f616415d6e0 0x7f6154004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 --2- 172.21.15.3:0/2325914612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6154004db0 0x7f6154005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61640cd320 con 0x7f6154004db0 2023-03-03T06:27:22.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615400bae0 con 0x7f6154003a50 2023-03-03T06:27:22.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.270+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615400bc70 con 0x7f616415d6e0 2023-03-03T06:27:22.279 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f6168bc1700 1 --2- 172.21.15.3:0/2325914612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6154003a50 0x7f6154003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f61693c2700 1 --2- 172.21.15.3:0/2325914612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6154004db0 0x7f6154005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f6163fff700 1 --2- 172.21.15.3:0/2325914612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f616415d6e0 0x7f6154004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f6168bc1700 1 -- 172.21.15.3:0/2325914612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f616415d6e0 msgr2=0x7f6154004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f6168bc1700 1 --2- 172.21.15.3:0/2325914612 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f616415d6e0 0x7f6154004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f6168bc1700 1 -- 172.21.15.3:0/2325914612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6154004db0 msgr2=0x7f6154005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:22.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f6168bc1700 1 --2- 172.21.15.3:0/2325914612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6154004db0 0x7f6154005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f6168bc1700 1 -- 172.21.15.3:0/2325914612 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61540035d0 con 0x7f6154003a50 2023-03-03T06:27:22.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f61693c2700 1 --2- 172.21.15.3:0/2325914612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6154004db0 0x7f6154005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:27:22.281 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f6168bc1700 1 --2- 172.21.15.3:0/2325914612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6154003a50 0x7f6154003e60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f615800c320 tx=0x7f61540058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeabf0167991aa5d server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158007c30 con 0x7f6154003a50 2023-03-03T06:27:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615800ce80 con 0x7f6154003a50 2023-03-03T06:27:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f61637fe700 1 -- 172.21.15.3:0/2325914612 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158005bc0 con 0x7f6154003a50 2023-03-03T06:27:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f6169bc3700 1 -- 172.21.15.3:0/2325914612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6154003a50 msgr2=0x7f6154003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f6169bc3700 1 --2- 172.21.15.3:0/2325914612 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6154003a50 0x7f6154003e60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f615800c320 tx=0x7f61540058a0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f61693c2700 1 -- 172.21.15.3:0/2325914612 reap_dead start 2023-03-03T06:27:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f6169bc3700 1 -- 172.21.15.3:0/2325914612 shutdown_connections 2023-03-03T06:27:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.274+0000 7f6169bc3700 1 -- 172.21.15.3:0/2325914612 >> 172.21.15.3:0/2325914612 conn(0x7f616406e5c0 msgr2=0x7f616406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.278+0000 7f6169bc3700 1 -- 172.21.15.3:0/2325914612 shutdown_connections 2023-03-03T06:27:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.278+0000 7f6169bc3700 1 -- 172.21.15.3:0/2325914612 wait complete. 2023-03-03T06:27:22.284 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.278+0000 7f6169bc3700 1 Processor -- start 2023-03-03T06:27:22.285 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.278+0000 7f6169bc3700 1 -- start start 2023-03-03T06:27:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f6169bc3700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6154003a50 0x7f61641f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f6169bc3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61640b3d90 0x7f61640b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f6169bc3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6164155060 0x7f61641554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f6169bc3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61640b6160 con 0x7f6164155060 2023-03-03T06:27:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f6169bc3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164159cd0 con 0x7f61640b3d90 2023-03-03T06:27:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f6169bc3700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61640b4740 con 0x7f6154003a50 2023-03-03T06:27:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f6163fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61640b3d90 0x7f61640b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f61693c2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6164155060 0x7f61641554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f6163fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61640b3d90 0x7f61640b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:36910/0 (socket says 172.21.15.3:36910) 2023-03-03T06:27:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f6163fff700 1 -- 172.21.15.3:0/2667910164 learned_addr learned my addr 172.21.15.3:0/2667910164 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f6168bc1700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6154003a50 0x7f61641f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f61693c2700 1 -- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6154003a50 msgr2=0x7f61641f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f61693c2700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6154003a50 0x7f61641f4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f61693c2700 1 -- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61640b3d90 msgr2=0x7f61640b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f61693c2700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61640b3d90 0x7f61640b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f61693c2700 1 -- 172.21.15.3:0/2667910164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640cef40 con 0x7f6164155060 2023-03-03T06:27:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f6163fff700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61640b3d90 0x7f61640b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:27:22.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.282+0000 7f61693c2700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6164155060 0x7f61641554d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f61640d0780 tx=0x7f615c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:22.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.286+0000 7f61617fa700 1 -- 172.21.15.3:0/2667910164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c003c30 con 0x7f6164155060 2023-03-03T06:27:22.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.286+0000 7f61617fa700 1 -- 172.21.15.3:0/2667910164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c003d90 con 0x7f6164155060 2023-03-03T06:27:22.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.286+0000 7f61617fa700 1 -- 172.21.15.3:0/2667910164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00db20 con 0x7f6164155060 2023-03-03T06:27:22.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.286+0000 7f6169bc3700 1 -- 172.21.15.3:0/2667910164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61640cd8e0 con 0x7f6164155060 2023-03-03T06:27:22.291 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.286+0000 7f6169bc3700 1 -- 172.21.15.3:0/2667910164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61640cf0d0 con 0x7f6164155060 2023-03-03T06:27:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.286+0000 7f6169bc3700 1 -- 172.21.15.3:0/2667910164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61641f54a0 con 0x7f6164155060 2023-03-03T06:27:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.290+0000 7f61617fa700 1 -- 172.21.15.3:0/2667910164 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f615c00dc80 con 0x7f6164155060 2023-03-03T06:27:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.294+0000 7f61617fa700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6148065030 0x7f61480674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.294+0000 7f61617fa700 1 -- 172.21.15.3:0/2667910164 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f615c082570 con 0x7f6164155060 2023-03-03T06:27:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.294+0000 7f6168bc1700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6148065030 0x7f61480674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.294+0000 7f6168bc1700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6148065030 0x7f61480674e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c00 tx=0x7f6158007000 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:22.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.298+0000 7f61617fa700 1 -- 172.21.15.3:0/2667910164 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f615c050330 con 0x7f6164155060 2023-03-03T06:27:22.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.494+0000 7f6169bc3700 1 -- 172.21.15.3:0/2667910164 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6164073e90 con 0x7f6148065030 2023-03-03T06:27:22.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.606+0000 7f61617fa700 1 -- 172.21.15.3:0/2667910164 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f6164073e90 con 0x7f6148065030 2023-03-03T06:27:22.613 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-03-03T06:27:22.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.610+0000 7f6169bc3700 1 -- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6148065030 msgr2=0x7f61480674e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.610+0000 7f6169bc3700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6148065030 0x7f61480674e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c00 tx=0x7f6158007000 comp rx=0 tx=0).stop 2023-03-03T06:27:22.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.610+0000 7f6169bc3700 1 -- 172.21.15.3:0/2667910164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6164155060 msgr2=0x7f61641554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.610+0000 7f6169bc3700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6164155060 0x7f61641554d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f61640d0780 tx=0x7f615c00d430 comp rx=0 tx=0).stop 2023-03-03T06:27:22.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.614+0000 7f6169bc3700 1 -- 172.21.15.3:0/2667910164 shutdown_connections 2023-03-03T06:27:22.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.614+0000 7f6169bc3700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6148065030 0x7f61480674e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.614+0000 7f6169bc3700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6164155060 0x7f61641554d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.619 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.614+0000 7f6169bc3700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61640b3d90 0x7f61640b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.614+0000 7f6169bc3700 1 --2- 172.21.15.3:0/2667910164 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6154003a50 0x7f61641f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.614+0000 7f6169bc3700 1 -- 172.21.15.3:0/2667910164 >> 172.21.15.3:0/2667910164 conn(0x7f616406e5c0 msgr2=0x7f616415f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:22.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.614+0000 7f6169bc3700 1 -- 172.21.15.3:0/2667910164 shutdown_connections 2023-03-03T06:27:22.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.614+0000 7f6169bc3700 1 -- 172.21.15.3:0/2667910164 wait complete. 2023-03-03T06:27:22.656 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-03-03T06:27:22.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir2 1M 2023-03-03T06:27:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.878+0000 7f699ad58700 1 Processor -- start 2023-03-03T06:27:22.885 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.878+0000 7f699ad58700 1 -- start start 2023-03-03T06:27:22.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.882+0000 7f699ad58700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6994159240 0x7f69940cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:22.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.882+0000 7f699ad58700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6994070020 con 0x7f69940ce150 2023-03-03T06:27:22.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.882+0000 7f699ad58700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6994070300 con 0x7f69940cdc50 2023-03-03T06:27:22.889 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.882+0000 7f699ad58700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699415dd40 con 0x7f6994159240 2023-03-03T06:27:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.886+0000 7f699a557700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f69940ce150 0x7f69940ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43074/0 (socket says 172.21.15.3:43074) 2023-03-03T06:27:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.886+0000 7f699a557700 1 -- 172.21.15.3:0/1771230557 learned_addr learned my addr 172.21.15.3:0/1771230557 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:22.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.886+0000 7f6999d56700 1 --2- 172.21.15.3:0/1771230557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6994159240 0x7f69940cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:22.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.886+0000 7f6998d54700 1 -- 172.21.15.3:0/1771230557 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027098267 0 0) 0x7f6994070020 con 0x7f69940ce150 2023-03-03T06:27:22.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.886+0000 7f6998d54700 1 -- 172.21.15.3:0/1771230557 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69800035d0 con 0x7f69940ce150 2023-03-03T06:27:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.886+0000 7f6998d54700 1 -- 172.21.15.3:0/1771230557 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176366088 0 0) 0x7f6994070300 con 0x7f69940cdc50 2023-03-03T06:27:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.886+0000 7f6999d56700 1 -- 172.21.15.3:0/1771230557 >> v1:172.21.15.32:6789/0 conn(0x7f69940cdc50 legacy=0x7f699415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.886+0000 7f6999d56700 1 -- 172.21.15.3:0/1771230557 >> v1:172.21.15.3:6789/0 conn(0x7f69940ce150 legacy=0x7f69940ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.886+0000 7f6999d56700 1 -- 172.21.15.3:0/1771230557 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699415ce30 con 0x7f6994159240 2023-03-03T06:27:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.886+0000 7f6999d56700 1 --2- 172.21.15.3:0/1771230557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6994159240 0x7f69940cd390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6990009870 tx=0x7f6990009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e9b7e11aa777151 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.886+0000 7f6998d54700 1 -- 172.21.15.3:0/1771230557 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990005e50 con 0x7f6994159240 2023-03-03T06:27:22.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.886+0000 7f6998d54700 1 -- 172.21.15.3:0/1771230557 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6990006030 con 0x7f6994159240 2023-03-03T06:27:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.886+0000 7f6998d54700 1 -- 172.21.15.3:0/1771230557 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69900063c0 con 0x7f6994159240 2023-03-03T06:27:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.890+0000 7f699ad58700 1 -- 172.21.15.3:0/1771230557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6994159240 msgr2=0x7f69940cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.890+0000 7f699ad58700 1 --2- 172.21.15.3:0/1771230557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6994159240 0x7f69940cd390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6990009870 tx=0x7f6990009b80 comp rx=0 tx=0).stop 2023-03-03T06:27:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.890+0000 7f699ad58700 1 -- 172.21.15.3:0/1771230557 shutdown_connections 2023-03-03T06:27:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.890+0000 7f699ad58700 1 --2- 172.21.15.3:0/1771230557 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f6994159240 0x7f69940cd390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.890+0000 7f699ad58700 1 -- 172.21.15.3:0/1771230557 >> 172.21.15.3:0/1771230557 conn(0x7f6994069d70 msgr2=0x7f699406a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.890+0000 7f699ad58700 1 -- 172.21.15.3:0/1771230557 shutdown_connections 2023-03-03T06:27:22.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.890+0000 7f699ad58700 1 -- 172.21.15.3:0/1771230557 wait complete. 2023-03-03T06:27:22.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.894+0000 7f699ad58700 1 Processor -- start 2023-03-03T06:27:22.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.894+0000 7f699ad58700 1 -- start start 2023-03-03T06:27:22.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.894+0000 7f699ad58700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69940cdc50 0x7f6994157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:22.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.894+0000 7f699ad58700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69940ce150 0x7f69940b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:22.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.894+0000 7f699ad58700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6994159240 0x7f69940c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:22.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.894+0000 7f699ad58700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6994157550 con 0x7f6994159240 2023-03-03T06:27:22.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.894+0000 7f699ad58700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6994159b60 con 0x7f69940cdc50 2023-03-03T06:27:22.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.894+0000 7f699ad58700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6994159e40 con 0x7f69940ce150 2023-03-03T06:27:22.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.894+0000 7f699a557700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6994159240 0x7f69940c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:22.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.894+0000 7f699a557700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6994159240 0x7f69940c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52656/0 (socket says 172.21.15.3:52656) 2023-03-03T06:27:22.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.894+0000 7f6999d56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69940cdc50 0x7f6994157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:22.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.894+0000 7f699a557700 1 -- 172.21.15.3:0/459918855 learned_addr learned my addr 172.21.15.3:0/459918855 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:22.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.894+0000 7f6999555700 1 --2- 172.21.15.3:0/459918855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69940ce150 0x7f69940b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:22.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.898+0000 7f699a557700 1 -- 172.21.15.3:0/459918855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69940ce150 msgr2=0x7f69940b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.898+0000 7f699a557700 1 --2- 172.21.15.3:0/459918855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69940ce150 0x7f69940b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.898+0000 7f699a557700 1 -- 172.21.15.3:0/459918855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69940cdc50 msgr2=0x7f6994157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.898+0000 7f699a557700 1 --2- 172.21.15.3:0/459918855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69940cdc50 0x7f6994157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.898+0000 7f699a557700 1 -- 172.21.15.3:0/459918855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6990009450 con 0x7f6994159240 2023-03-03T06:27:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.898+0000 7f699a557700 1 --2- 172.21.15.3:0/459918855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6994159240 0x7f69940c8990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f698c00cb10 tx=0x7f698c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.898+0000 7f698a7fc700 1 -- 172.21.15.3:0/459918855 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c007480 con 0x7f6994159240 2023-03-03T06:27:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.898+0000 7f698a7fc700 1 -- 172.21.15.3:0/459918855 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698c0075e0 con 0x7f6994159240 2023-03-03T06:27:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.898+0000 7f698a7fc700 1 -- 172.21.15.3:0/459918855 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c007970 con 0x7f6994159240 2023-03-03T06:27:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.898+0000 7f699ad58700 1 -- 172.21.15.3:0/459918855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6994159610 con 0x7f6994159240 2023-03-03T06:27:22.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.898+0000 7f699ad58700 1 -- 172.21.15.3:0/459918855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699415d110 con 0x7f6994159240 2023-03-03T06:27:22.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.898+0000 7f699ad58700 1 -- 172.21.15.3:0/459918855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69940705e0 con 0x7f6994159240 2023-03-03T06:27:22.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.902+0000 7f698a7fc700 1 -- 172.21.15.3:0/459918855 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f698c019030 con 0x7f6994159240 2023-03-03T06:27:22.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.902+0000 7f698a7fc700 1 --2- 172.21.15.3:0/459918855 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6978065030 0x7f69780674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:22.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.902+0000 7f698a7fc700 1 -- 172.21.15.3:0/459918855 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f698c081ac0 con 0x7f6994159240 2023-03-03T06:27:22.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.906+0000 7f6999d56700 1 --2- 172.21.15.3:0/459918855 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6978065030 0x7f69780674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:22.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.906+0000 7f6999d56700 1 --2- 172.21.15.3:0/459918855 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6978065030 0x7f69780674e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6990013040 tx=0x7f6990008070 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:22.910+0000 7f698a7fc700 1 -- 172.21.15.3:0/459918855 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f698c04f880 con 0x7f6994159240 2023-03-03T06:27:23.116 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.110+0000 7f699ad58700 1 -- 172.21.15.3:0/459918855 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6994073e90 con 0x7f6978065030 2023-03-03T06:27:23.228 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.222+0000 7f698a7fc700 1 -- 172.21.15.3:0/459918855 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f6994073e90 con 0x7f6978065030 2023-03-03T06:27:23.229 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-03-03T06:27:23.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.226+0000 7f699ad58700 1 -- 172.21.15.3:0/459918855 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6978065030 msgr2=0x7f69780674e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:23.233 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.226+0000 7f699ad58700 1 --2- 172.21.15.3:0/459918855 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6978065030 0x7f69780674e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6990013040 tx=0x7f6990008070 comp rx=0 tx=0).stop 2023-03-03T06:27:23.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.226+0000 7f699ad58700 1 -- 172.21.15.3:0/459918855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6994159240 msgr2=0x7f69940c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:23.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.226+0000 7f699ad58700 1 --2- 172.21.15.3:0/459918855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6994159240 0x7f69940c8990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f698c00cb10 tx=0x7f698c00cb40 comp rx=0 tx=0).stop 2023-03-03T06:27:23.234 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.226+0000 7f699ad58700 1 -- 172.21.15.3:0/459918855 shutdown_connections 2023-03-03T06:27:23.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.226+0000 7f699ad58700 1 --2- 172.21.15.3:0/459918855 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f6978065030 0x7f69780674e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:23.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.226+0000 7f699ad58700 1 --2- 172.21.15.3:0/459918855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6994159240 0x7f69940c8990 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:23.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.226+0000 7f699ad58700 1 --2- 172.21.15.3:0/459918855 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f69940ce150 0x7f69940b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:23.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.226+0000 7f699ad58700 1 --2- 172.21.15.3:0/459918855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69940cdc50 0x7f6994157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:23.235 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.226+0000 7f699ad58700 1 -- 172.21.15.3:0/459918855 >> 172.21.15.3:0/459918855 conn(0x7f6994069d70 msgr2=0x7f69940cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:23.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.230+0000 7f699ad58700 1 -- 172.21.15.3:0/459918855 shutdown_connections 2023-03-03T06:27:23.236 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.230+0000 7f699ad58700 1 -- 172.21.15.3:0/459918855 wait complete. 2023-03-03T06:27:23.267 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-03-03T06:27:23.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir3 1M 2023-03-03T06:27:23.501 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.494+0000 7fcaba812700 1 Processor -- start 2023-03-03T06:27:23.502 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.498+0000 7fcaba812700 1 -- start start 2023-03-03T06:27:23.506 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.502+0000 7fcaba812700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcab4156ba0 0x7fcab4071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:23.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.502+0000 7fcaba812700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab40cf3b0 con 0x7fcab40d0b30 2023-03-03T06:27:23.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.502+0000 7fcaba812700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab4156f70 con 0x7fcab415d6e0 2023-03-03T06:27:23.507 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.502+0000 7fcaba812700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40cd320 con 0x7fcab4156ba0 2023-03-03T06:27:23.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.502+0000 7fcab9810700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcab4156ba0 0x7fcab4071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:23.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.502+0000 7fcaba011700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcab40d0b30 0x7fcab40b46b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43076/0 (socket says 172.21.15.3:43076) 2023-03-03T06:27:23.509 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.502+0000 7fcaba011700 1 -- 172.21.15.3:0/3345639109 learned_addr learned my addr 172.21.15.3:0/3345639109 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:23.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.502+0000 7fcab880e700 1 -- 172.21.15.3:0/3345639109 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994977667 0 0) 0x7fcab40cf3b0 con 0x7fcab40d0b30 2023-03-03T06:27:23.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.502+0000 7fcab880e700 1 -- 172.21.15.3:0/3345639109 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaa00035d0 con 0x7fcab40d0b30 2023-03-03T06:27:23.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.502+0000 7fcab9810700 1 -- 172.21.15.3:0/3345639109 >> v1:172.21.15.32:6789/0 conn(0x7fcab415d6e0 legacy=0x7fcab40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:23.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.502+0000 7fcab9810700 1 -- 172.21.15.3:0/3345639109 >> v1:172.21.15.3:6789/0 conn(0x7fcab40d0b30 legacy=0x7fcab40b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:23.510 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.502+0000 7fcab9810700 1 -- 172.21.15.3:0/3345639109 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab40cd600 con 0x7fcab4156ba0 2023-03-03T06:27:23.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.506+0000 7fcab9810700 1 --2- 172.21.15.3:0/3345639109 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcab4156ba0 0x7fcab4071280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009880 tx=0x7fcab0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71d1a8202fa4b63e server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:23.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.506+0000 7fcab880e700 1 -- 172.21.15.3:0/3345639109 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0011830 con 0x7fcab4156ba0 2023-03-03T06:27:23.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.506+0000 7fcab880e700 1 -- 172.21.15.3:0/3345639109 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab0011990 con 0x7fcab4156ba0 2023-03-03T06:27:23.511 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.506+0000 7fcab880e700 1 -- 172.21.15.3:0/3345639109 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0011d20 con 0x7fcab4156ba0 2023-03-03T06:27:23.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.506+0000 7fcaba812700 1 -- 172.21.15.3:0/3345639109 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcab4156ba0 msgr2=0x7fcab4071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:23.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.506+0000 7fcaba812700 1 --2- 172.21.15.3:0/3345639109 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcab4156ba0 0x7fcab4071280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009880 tx=0x7fcab0009b90 comp rx=0 tx=0).stop 2023-03-03T06:27:23.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.506+0000 7fcaba812700 1 -- 172.21.15.3:0/3345639109 shutdown_connections 2023-03-03T06:27:23.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.506+0000 7fcaba812700 1 --2- 172.21.15.3:0/3345639109 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcab4156ba0 0x7fcab4071280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:23.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.506+0000 7fcaba812700 1 -- 172.21.15.3:0/3345639109 >> 172.21.15.3:0/3345639109 conn(0x7fcab406e5c0 msgr2=0x7fcab406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:23.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.506+0000 7fcaba812700 1 -- 172.21.15.3:0/3345639109 shutdown_connections 2023-03-03T06:27:23.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.506+0000 7fcaba812700 1 -- 172.21.15.3:0/3345639109 wait complete. 2023-03-03T06:27:23.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.510+0000 7fcaba812700 1 Processor -- start 2023-03-03T06:27:23.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.510+0000 7fcaba812700 1 -- start start 2023-03-03T06:27:23.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcaba812700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab40d0b30 0x7fcab41f1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:23.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcaba812700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcab4156ba0 0x7fcab40b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:23.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcaba812700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab415d6e0 0x7fcab40c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:23.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcaba812700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40b4a20 con 0x7fcab40d0b30 2023-03-03T06:27:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcaba812700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40b41c0 con 0x7fcab415d6e0 2023-03-03T06:27:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcaba812700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab41f1970 con 0x7fcab4156ba0 2023-03-03T06:27:23.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcab9810700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab40d0b30 0x7fcab41f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:23.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcab9810700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab40d0b30 0x7fcab41f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52672/0 (socket says 172.21.15.3:52672) 2023-03-03T06:27:23.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcab9810700 1 -- 172.21.15.3:0/1368876786 learned_addr learned my addr 172.21.15.3:0/1368876786 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:23.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcab900f700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcab4156ba0 0x7fcab40b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:23.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcaba011700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab415d6e0 0x7fcab40c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:23.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcab9810700 1 -- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcab4156ba0 msgr2=0x7fcab40b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:23.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcab9810700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcab4156ba0 0x7fcab40b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:23.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcab9810700 1 -- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab415d6e0 msgr2=0x7fcab40c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:23.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcab9810700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab415d6e0 0x7fcab40c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:23.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcab9810700 1 -- 172.21.15.3:0/1368876786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00094b0 con 0x7fcab40d0b30 2023-03-03T06:27:23.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcaba011700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab415d6e0 0x7fcab40c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:27:23.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcab9810700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab40d0b30 0x7fcab41f1430 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcab0013040 tx=0x7fcab0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcaaa7fc700 1 -- 172.21.15.3:0/1368876786 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab00036c0 con 0x7fcab40d0b30 2023-03-03T06:27:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcaaa7fc700 1 -- 172.21.15.3:0/1368876786 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab0003820 con 0x7fcab40d0b30 2023-03-03T06:27:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcaba812700 1 -- 172.21.15.3:0/1368876786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab40717c0 con 0x7fcab40d0b30 2023-03-03T06:27:23.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcaaa7fc700 1 -- 172.21.15.3:0/1368876786 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0003b90 con 0x7fcab40d0b30 2023-03-03T06:27:23.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.514+0000 7fcaba812700 1 -- 172.21.15.3:0/1368876786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab40cd8e0 con 0x7fcab40d0b30 2023-03-03T06:27:23.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.518+0000 7fcaba812700 1 -- 172.21.15.3:0/1368876786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab40cff50 con 0x7fcab40d0b30 2023-03-03T06:27:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.518+0000 7fcaaa7fc700 1 -- 172.21.15.3:0/1368876786 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7fcab000f050 con 0x7fcab40d0b30 2023-03-03T06:27:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.522+0000 7fcaaa7fc700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fca98065000 0x7fca980674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:23.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.522+0000 7fcaaa7fc700 1 -- 172.21.15.3:0/1368876786 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fcab0083660 con 0x7fcab40d0b30 2023-03-03T06:27:23.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.522+0000 7fcab900f700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fca98065000 0x7fca980674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:23.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.526+0000 7fcab900f700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fca98065000 0x7fca980674b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcab41552e0 tx=0x7fcaa4009040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:23.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.526+0000 7fcaaa7fc700 1 -- 172.21.15.3:0/1368876786 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fcab00514f0 con 0x7fcab40d0b30 2023-03-03T06:27:23.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.726+0000 7fcaba812700 1 -- 172.21.15.3:0/1368876786 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcab4073e90 con 0x7fca98065000 2023-03-03T06:27:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.838+0000 7fcaaa7fc700 1 -- 172.21.15.3:0/1368876786 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fcab4073e90 con 0x7fca98065000 2023-03-03T06:27:23.844 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-03-03T06:27:23.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.842+0000 7fcaba812700 1 -- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fca98065000 msgr2=0x7fca980674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:23.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.842+0000 7fcaba812700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fca98065000 0x7fca980674b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcab41552e0 tx=0x7fcaa4009040 comp rx=0 tx=0).stop 2023-03-03T06:27:23.848 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.842+0000 7fcaba812700 1 -- 172.21.15.3:0/1368876786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab40d0b30 msgr2=0x7fcab41f1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.842+0000 7fcaba812700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab40d0b30 0x7fcab41f1430 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcab0013040 tx=0x7fcab0004770 comp rx=0 tx=0).stop 2023-03-03T06:27:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.842+0000 7fcaba812700 1 -- 172.21.15.3:0/1368876786 shutdown_connections 2023-03-03T06:27:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.842+0000 7fcaba812700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fca98065000 0x7fca980674b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.842+0000 7fcaba812700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcab415d6e0 0x7fcab40c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.842+0000 7fcaba812700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fcab4156ba0 0x7fcab40b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.842+0000 7fcaba812700 1 --2- 172.21.15.3:0/1368876786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcab40d0b30 0x7fcab41f1430 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.842+0000 7fcaba812700 1 -- 172.21.15.3:0/1368876786 >> 172.21.15.3:0/1368876786 conn(0x7fcab406e5c0 msgr2=0x7fcab415cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:23.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.842+0000 7fcaba812700 1 -- 172.21.15.3:0/1368876786 shutdown_connections 2023-03-03T06:27:23.851 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:23.842+0000 7fcaba812700 1 -- 172.21.15.3:0/1368876786 wait complete. 2023-03-03T06:27:23.883 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-03-03T06:27:23.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir4 1M 2023-03-03T06:27:24.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.094+0000 7f505b59e700 1 Processor -- start 2023-03-03T06:27:24.100 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.094+0000 7f505b59e700 1 -- start start 2023-03-03T06:27:24.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.098+0000 7f505b59e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f505c15fb60 0x7f505c160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:24.104 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.098+0000 7f505b59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f505c0c95c0 0x7f505c06d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.098+0000 7f505b59e700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f505c0c9a10 0x7f505c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:24.105 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.098+0000 7f505b59e700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c15ff30 con 0x7f505c0c9a10 2023-03-03T06:27:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.098+0000 7f505b59e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c0ccf20 con 0x7f505c15fb60 2023-03-03T06:27:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.098+0000 7f505b59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0cd200 con 0x7f505c0c95c0 2023-03-03T06:27:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f505c15fb60 0x7f505c160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505a59c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f505c15fb60 0x7f505c160ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:36932/0 (socket says 172.21.15.3:36932) 2023-03-03T06:27:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505a59c700 1 -- 172.21.15.3:0/2938751078 learned_addr learned my addr 172.21.15.3:0/2938751078 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:24.106 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f5059d9b700 1 --2- 172.21.15.3:0/2938751078 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f505c0c95c0 0x7f505c06d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505ad9d700 1 --2- 172.21.15.3:0/2938751078 >> v2:172.21.15.3:3300/0 conn(0x7f505c0c9a10 0x7f505c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505a59c700 1 -- 172.21.15.3:0/2938751078 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f505c0c95c0 msgr2=0x7f505c06d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505a59c700 1 --2- 172.21.15.3:0/2938751078 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f505c0c95c0 0x7f505c06d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505a59c700 1 -- 172.21.15.3:0/2938751078 >> v2:172.21.15.3:3300/0 conn(0x7f505c0c9a10 msgr2=0x7f505c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:24.107 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505a59c700 1 --2- 172.21.15.3:0/2938751078 >> v2:172.21.15.3:3300/0 conn(0x7f505c0c9a10 0x7f505c06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505a59c700 1 -- 172.21.15.3:0/2938751078 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c0cd4e0 con 0x7f505c15fb60 2023-03-03T06:27:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505ad9d700 1 --2- 172.21.15.3:0/2938751078 >> v2:172.21.15.3:3300/0 conn(0x7f505c0c9a10 0x7f505c06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:27:24.108 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505a59c700 1 --2- 172.21.15.3:0/2938751078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c15fb60 0x7f505c160ac0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f505c06c7a0 tx=0x7f504c009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3022ddf1c1fd5a server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:24.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505959a700 1 -- 172.21.15.3:0/2938751078 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c011830 con 0x7f505c15fb60 2023-03-03T06:27:24.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505959a700 1 -- 172.21.15.3:0/2938751078 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c011990 con 0x7f505c15fb60 2023-03-03T06:27:24.109 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.102+0000 7f505959a700 1 -- 172.21.15.3:0/2938751078 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c011d20 con 0x7f505c15fb60 2023-03-03T06:27:24.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.106+0000 7f505b59e700 1 -- 172.21.15.3:0/2938751078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c15fb60 msgr2=0x7f505c160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:24.110 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.106+0000 7f505b59e700 1 --2- 172.21.15.3:0/2938751078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c15fb60 0x7f505c160ac0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f505c06c7a0 tx=0x7f504c009bf0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.111 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.106+0000 7f505b59e700 1 -- 172.21.15.3:0/2938751078 shutdown_connections 2023-03-03T06:27:24.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.106+0000 7f505b59e700 1 --2- 172.21.15.3:0/2938751078 >> v2:172.21.15.3:3300/0 conn(0x7f505c0c9a10 0x7f505c06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.106+0000 7f505b59e700 1 --2- 172.21.15.3:0/2938751078 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f505c0c95c0 0x7f505c06d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.106+0000 7f505b59e700 1 --2- 172.21.15.3:0/2938751078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c15fb60 0x7f505c160ac0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.106+0000 7f505b59e700 1 -- 172.21.15.3:0/2938751078 >> 172.21.15.3:0/2938751078 conn(0x7f505c0cfc90 msgr2=0x7f505c0cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:24.112 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.106+0000 7f505b59e700 1 -- 172.21.15.3:0/2938751078 shutdown_connections 2023-03-03T06:27:24.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.106+0000 7f505b59e700 1 -- 172.21.15.3:0/2938751078 wait complete. 2023-03-03T06:27:24.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.110+0000 7f505b59e700 1 Processor -- start 2023-03-03T06:27:24.115 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.110+0000 7f505b59e700 1 -- start start 2023-03-03T06:27:24.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.110+0000 7f505b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c0c95c0 0x7f505c1593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:24.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.110+0000 7f505b59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f505c0c9a10 0x7f505c0ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:24.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.110+0000 7f505b59e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f505c15fb60 0x7f505c0c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:24.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.110+0000 7f505b59e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0ad9a0 con 0x7f505c0c9a10 2023-03-03T06:27:24.117 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.110+0000 7f505b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0acd20 con 0x7f505c0c95c0 2023-03-03T06:27:24.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.110+0000 7f505b59e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0ad000 con 0x7f505c15fb60 2023-03-03T06:27:24.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f5059d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f505c0c9a10 0x7f505c0ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:24.118 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f5059d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f505c0c9a10 0x7f505c0ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52692/0 (socket says 172.21.15.3:52692) 2023-03-03T06:27:24.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f5059d9b700 1 -- 172.21.15.3:0/3885037140 learned_addr learned my addr 172.21.15.3:0/3885037140 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:24.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f505a59c700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c0c95c0 0x7f505c1593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:24.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f505ad9d700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f505c15fb60 0x7f505c0c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:24.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f5059d9b700 1 -- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f505c15fb60 msgr2=0x7f505c0c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:24.119 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f5059d9b700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f505c15fb60 0x7f505c0c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f5059d9b700 1 -- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c0c95c0 msgr2=0x7f505c1593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:24.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f5059d9b700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c0c95c0 0x7f505c1593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.120 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f5059d9b700 1 -- 172.21.15.3:0/3885037140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c009570 con 0x7f505c0c9a10 2023-03-03T06:27:24.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f5059d9b700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f505c0c9a10 0x7f505c0ac2d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f505000e870 tx=0x7f505000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:24.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f504affd700 1 -- 172.21.15.3:0/3885037140 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500095c0 con 0x7f505c0c9a10 2023-03-03T06:27:24.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f504affd700 1 -- 172.21.15.3:0/3885037140 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5050013680 con 0x7f505c0c9a10 2023-03-03T06:27:24.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f504affd700 1 -- 172.21.15.3:0/3885037140 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050013a10 con 0x7f505c0c9a10 2023-03-03T06:27:24.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f505b59e700 1 -- 172.21.15.3:0/3885037140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c06cb80 con 0x7f505c0c9a10 2023-03-03T06:27:24.121 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.114+0000 7f505b59e700 1 -- 172.21.15.3:0/3885037140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c06c8a0 con 0x7f505c0c9a10 2023-03-03T06:27:24.122 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.118+0000 7f505b59e700 1 -- 172.21.15.3:0/3885037140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c0adae0 con 0x7f505c0c9a10 2023-03-03T06:27:24.125 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.118+0000 7f504affd700 1 -- 172.21.15.3:0/3885037140 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f5050019030 con 0x7f505c0c9a10 2023-03-03T06:27:24.127 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.122+0000 7f504affd700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f503c065030 0x7f503c0674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:24.128 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.122+0000 7f504affd700 1 -- 172.21.15.3:0/3885037140 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f5050082070 con 0x7f505c0c9a10 2023-03-03T06:27:24.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.122+0000 7f505a59c700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f503c065030 0x7f503c0674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:24.130 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.126+0000 7f505a59c700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f503c065030 0x7f503c0674e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f504c009c30 tx=0x7f504c008030 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:24.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.130+0000 7f504affd700 1 -- 172.21.15.3:0/3885037140 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f505004fdd0 con 0x7f505c0c9a10 2023-03-03T06:27:24.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.330+0000 7f505b59e700 1 -- 172.21.15.3:0/3885037140 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f505c0c3020 con 0x7f503c065030 2023-03-03T06:27:24.452 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.446+0000 7f504affd700 1 -- 172.21.15.3:0/3885037140 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f505c0c3020 con 0x7f503c065030 2023-03-03T06:27:24.453 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-03-03T06:27:24.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.450+0000 7f505b59e700 1 -- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f503c065030 msgr2=0x7f503c0674e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:24.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.450+0000 7f505b59e700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f503c065030 0x7f503c0674e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f504c009c30 tx=0x7f504c008030 comp rx=0 tx=0).stop 2023-03-03T06:27:24.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.450+0000 7f505b59e700 1 -- 172.21.15.3:0/3885037140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f505c0c9a10 msgr2=0x7f505c0ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:24.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.450+0000 7f505b59e700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f505c0c9a10 0x7f505c0ac2d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f505000e870 tx=0x7f505000eb80 comp rx=0 tx=0).stop 2023-03-03T06:27:24.458 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.450+0000 7f505b59e700 1 -- 172.21.15.3:0/3885037140 shutdown_connections 2023-03-03T06:27:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.450+0000 7f505b59e700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f503c065030 0x7f503c0674e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.450+0000 7f505b59e700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f505c15fb60 0x7f505c0c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.450+0000 7f505b59e700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f505c0c9a10 0x7f505c0ac2d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.450+0000 7f505b59e700 1 --2- 172.21.15.3:0/3885037140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f505c0c95c0 0x7f505c1593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.450+0000 7f505b59e700 1 -- 172.21.15.3:0/3885037140 >> 172.21.15.3:0/3885037140 conn(0x7f505c0cfc90 msgr2=0x7f505c0cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.454+0000 7f505b59e700 1 -- 172.21.15.3:0/3885037140 shutdown_connections 2023-03-03T06:27:24.459 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.454+0000 7f505b59e700 1 -- 172.21.15.3:0/3885037140 wait complete. 2023-03-03T06:27:24.492 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-03-03T06:27:24.493 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir5 1M 2023-03-03T06:27:24.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.710+0000 7fc4eb390700 1 Processor -- start 2023-03-03T06:27:24.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.710+0000 7fc4eb390700 1 -- start start 2023-03-03T06:27:24.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4eb390700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4e415d720 0x7fc4e41608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:24.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4eb390700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4e40cf610 0x7fc4e40cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:24.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4eb390700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e41600f0 con 0x7fc4e4156cc0 2023-03-03T06:27:24.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4eb390700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e40cff70 con 0x7fc4e415d720 2023-03-03T06:27:24.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4eb390700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40cd420 con 0x7fc4e40cf610 2023-03-03T06:27:24.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4e9b8d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4e415d720 0x7fc4e41608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:24.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4e9b8d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc4e415d720 0x7fc4e41608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52694/0 (socket says 172.21.15.3:52694) 2023-03-03T06:27:24.721 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4e9b8d700 1 -- 172.21.15.3:0/3486078157 learned_addr learned my addr 172.21.15.3:0/3486078157 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:24.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4eab8f700 1 --2- 172.21.15.3:0/3486078157 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4e40cf610 0x7fc4e40cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:24.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4e9b8d700 1 -- 172.21.15.3:0/3486078157 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4e40cf610 msgr2=0x7fc4e40cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:24.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4e9b8d700 1 --2- 172.21.15.3:0/3486078157 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4e40cf610 0x7fc4e40cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4e9b8d700 1 -- 172.21.15.3:0/3486078157 >> v1:172.21.15.32:6789/0 conn(0x7fc4e4156cc0 legacy=0x7fc4e40713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:24.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4e9b8d700 1 -- 172.21.15.3:0/3486078157 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e40cd700 con 0x7fc4e415d720 2023-03-03T06:27:24.722 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.714+0000 7fc4e938c700 1 -- 172.21.15.3:0/3486078157 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271179236 0 0) 0x7fc4e41600f0 con 0x7fc4e4156cc0 2023-03-03T06:27:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.718+0000 7fc4e9b8d700 1 --2- 172.21.15.3:0/3486078157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e415d720 0x7fc4e41608b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009910 tx=0x7fc4d4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a50be5c4cd7bce server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.718+0000 7fc4e938c700 1 -- 172.21.15.3:0/3486078157 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4011830 con 0x7fc4e415d720 2023-03-03T06:27:24.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.718+0000 7fc4e938c700 1 -- 172.21.15.3:0/3486078157 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d4011990 con 0x7fc4e415d720 2023-03-03T06:27:24.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.718+0000 7fc4e938c700 1 -- 172.21.15.3:0/3486078157 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4011d40 con 0x7fc4e415d720 2023-03-03T06:27:24.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.718+0000 7fc4eb390700 1 -- 172.21.15.3:0/3486078157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e415d720 msgr2=0x7fc4e41608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:24.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.718+0000 7fc4eb390700 1 --2- 172.21.15.3:0/3486078157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e415d720 0x7fc4e41608b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009910 tx=0x7fc4d4009c20 comp rx=0 tx=0).stop 2023-03-03T06:27:24.724 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.718+0000 7fc4eb390700 1 -- 172.21.15.3:0/3486078157 shutdown_connections 2023-03-03T06:27:24.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.718+0000 7fc4eb390700 1 --2- 172.21.15.3:0/3486078157 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4e40cf610 0x7fc4e40cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.718+0000 7fc4eb390700 1 --2- 172.21.15.3:0/3486078157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e415d720 0x7fc4e41608b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.718+0000 7fc4eb390700 1 -- 172.21.15.3:0/3486078157 >> 172.21.15.3:0/3486078157 conn(0x7fc4e406e5c0 msgr2=0x7fc4e406e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:24.725 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.718+0000 7fc4eb390700 1 -- 172.21.15.3:0/3486078157 shutdown_connections 2023-03-03T06:27:24.726 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.718+0000 7fc4eb390700 1 -- 172.21.15.3:0/3486078157 wait complete. 2023-03-03T06:27:24.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.722+0000 7fc4eb390700 1 Processor -- start 2023-03-03T06:27:24.728 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.722+0000 7fc4eb390700 1 -- start start 2023-03-03T06:27:24.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4eb390700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e40cf610 0x7fc4e40c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:24.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4eb390700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e4156cc0 0x7fc4e40d2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:24.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4eb390700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4e415d720 0x7fc4e41530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:24.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4eb390700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40d2f20 con 0x7fc4e40cf610 2023-03-03T06:27:24.730 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4eb390700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40c9da0 con 0x7fc4e4156cc0 2023-03-03T06:27:24.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4eb390700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40ca080 con 0x7fc4e415d720 2023-03-03T06:27:24.731 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4ea38e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e40cf610 0x7fc4e40c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:24.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4ea38e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e40cf610 0x7fc4e40c5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52698/0 (socket says 172.21.15.3:52698) 2023-03-03T06:27:24.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4ea38e700 1 -- 172.21.15.3:0/4283438185 learned_addr learned my addr 172.21.15.3:0/4283438185 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:24.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4e9b8d700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e4156cc0 0x7fc4e40d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:24.732 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4eab8f700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4e415d720 0x7fc4e41530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:24.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4ea38e700 1 -- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4e415d720 msgr2=0x7fc4e41530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:24.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4ea38e700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4e415d720 0x7fc4e41530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4ea38e700 1 -- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e4156cc0 msgr2=0x7fc4e40d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:24.733 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4ea38e700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e4156cc0 0x7fc4e40d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:24.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4ea38e700 1 -- 172.21.15.3:0/4283438185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d4009570 con 0x7fc4e40cf610 2023-03-03T06:27:24.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4eab8f700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4e415d720 0x7fc4e41530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:27:24.734 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4ea38e700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e40cf610 0x7fc4e40c5c50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c520 tx=0x7fc4e000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:24.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4daffd700 1 -- 172.21.15.3:0/4283438185 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00054e0 con 0x7fc4e40cf610 2023-03-03T06:27:24.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4daffd700 1 -- 172.21.15.3:0/4283438185 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e00036a0 con 0x7fc4e40cf610 2023-03-03T06:27:24.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4eb390700 1 -- 172.21.15.3:0/4283438185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e40cd9e0 con 0x7fc4e40cf610 2023-03-03T06:27:24.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4daffd700 1 -- 172.21.15.3:0/4283438185 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0003a50 con 0x7fc4e40cf610 2023-03-03T06:27:24.735 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.726+0000 7fc4eb390700 1 -- 172.21.15.3:0/4283438185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e4160390 con 0x7fc4e40cf610 2023-03-03T06:27:24.736 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.730+0000 7fc4eb390700 1 -- 172.21.15.3:0/4283438185 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e4160df0 con 0x7fc4e40cf610 2023-03-03T06:27:24.737 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.730+0000 7fc4daffd700 1 -- 172.21.15.3:0/4283438185 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7fc4e0019030 con 0x7fc4e40cf610 2023-03-03T06:27:24.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.734+0000 7fc4daffd700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc4c8065000 0x7fc4c80674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:24.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.734+0000 7fc4daffd700 1 -- 172.21.15.3:0/4283438185 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fc4e0081620 con 0x7fc4e40cf610 2023-03-03T06:27:24.740 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.734+0000 7fc4e9b8d700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc4c8065000 0x7fc4c80674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:24.742 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.738+0000 7fc4e9b8d700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc4c8065000 0x7fc4c80674b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc4e415db80 tx=0x7fc4d4008030 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:24.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.742+0000 7fc4daffd700 1 -- 172.21.15.3:0/4283438185 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc4e004f3e0 con 0x7fc4e40cf610 2023-03-03T06:27:24.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:24.938+0000 7fc4eb390700 1 -- 172.21.15.3:0/4283438185 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4e4070ee0 con 0x7fc4c8065000 2023-03-03T06:27:25.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.046+0000 7fc4daffd700 1 -- 172.21.15.3:0/4283438185 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc4e4070ee0 con 0x7fc4c8065000 2023-03-03T06:27:25.053 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-03-03T06:27:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.050+0000 7fc4eb390700 1 -- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc4c8065000 msgr2=0x7fc4c80674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.050+0000 7fc4eb390700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc4c8065000 0x7fc4c80674b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc4e415db80 tx=0x7fc4d4008030 comp rx=0 tx=0).stop 2023-03-03T06:27:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.050+0000 7fc4eb390700 1 -- 172.21.15.3:0/4283438185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e40cf610 msgr2=0x7fc4e40c5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.050+0000 7fc4eb390700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e40cf610 0x7fc4e40c5c50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c520 tx=0x7fc4e000c830 comp rx=0 tx=0).stop 2023-03-03T06:27:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.050+0000 7fc4eb390700 1 -- 172.21.15.3:0/4283438185 shutdown_connections 2023-03-03T06:27:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.050+0000 7fc4eb390700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc4c8065000 0x7fc4c80674b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.050+0000 7fc4eb390700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc4e415d720 0x7fc4e41530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.050+0000 7fc4eb390700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e4156cc0 0x7fc4e40d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.050+0000 7fc4eb390700 1 --2- 172.21.15.3:0/4283438185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4e40cf610 0x7fc4e40c5c50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.050+0000 7fc4eb390700 1 -- 172.21.15.3:0/4283438185 >> 172.21.15.3:0/4283438185 conn(0x7fc4e406e5c0 msgr2=0x7fc4e415ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.050+0000 7fc4eb390700 1 -- 172.21.15.3:0/4283438185 shutdown_connections 2023-03-03T06:27:25.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.050+0000 7fc4eb390700 1 -- 172.21.15.3:0/4283438185 wait complete. 2023-03-03T06:27:25.091 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-03-03T06:27:25.091 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir6 1M 2023-03-03T06:27:25.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.298+0000 7fa6754fd700 1 Processor -- start 2023-03-03T06:27:25.303 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.298+0000 7fa6754fd700 1 -- start start 2023-03-03T06:27:25.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa6754fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6700cb9a0 0x7fa6700cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:25.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa6754fd700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa67015c230 con 0x7fa6700710c0 2023-03-03T06:27:25.306 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa6754fd700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6700cd130 con 0x7fa67006a780 2023-03-03T06:27:25.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa6754fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700caac0 con 0x7fa6700cb9a0 2023-03-03T06:27:25.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa66f7fe700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa6700710c0 0x7fa6700ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43084/0 (socket says 172.21.15.3:43084) 2023-03-03T06:27:25.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa66f7fe700 1 -- 172.21.15.3:0/934447842 learned_addr learned my addr 172.21.15.3:0/934447842 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:25.307 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa674cfc700 1 --2- 172.21.15.3:0/934447842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6700cb9a0 0x7fa6700cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa66effd700 1 -- 172.21.15.3:0/934447842 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263134965 0 0) 0x7fa67015c230 con 0x7fa6700710c0 2023-03-03T06:27:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa66effd700 1 -- 172.21.15.3:0/934447842 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6580057e0 con 0x7fa6700710c0 2023-03-03T06:27:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa66effd700 1 -- 172.21.15.3:0/934447842 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826767526 0 0) 0x7fa6700cd130 con 0x7fa67006a780 2023-03-03T06:27:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa66effd700 1 -- 172.21.15.3:0/934447842 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa67015c230 con 0x7fa67006a780 2023-03-03T06:27:25.308 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa674cfc700 1 -- 172.21.15.3:0/934447842 >> v1:172.21.15.32:6789/0 conn(0x7fa67006a780 legacy=0x7fa6700709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa674cfc700 1 -- 172.21.15.3:0/934447842 >> v1:172.21.15.3:6789/0 conn(0x7fa6700710c0 legacy=0x7fa6700ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa674cfc700 1 -- 172.21.15.3:0/934447842 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700cbf60 con 0x7fa6700cb9a0 2023-03-03T06:27:25.309 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.302+0000 7fa674cfc700 1 --2- 172.21.15.3:0/934447842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6700cb9a0 0x7fa6700cace0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa668009dc0 tx=0x7fa66800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5f1bc81ab597957 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:25.310 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.306+0000 7fa66effd700 1 -- 172.21.15.3:0/934447842 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800e070 con 0x7fa6700cb9a0 2023-03-03T06:27:25.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.306+0000 7fa66effd700 1 -- 172.21.15.3:0/934447842 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66800bb20 con 0x7fa6700cb9a0 2023-03-03T06:27:25.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.306+0000 7fa66effd700 1 -- 172.21.15.3:0/934447842 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668012430 con 0x7fa6700cb9a0 2023-03-03T06:27:25.311 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.306+0000 7fa6754fd700 1 -- 172.21.15.3:0/934447842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6700cb9a0 msgr2=0x7fa6700cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.306+0000 7fa6754fd700 1 --2- 172.21.15.3:0/934447842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6700cb9a0 0x7fa6700cace0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa668009dc0 tx=0x7fa66800b330 comp rx=0 tx=0).stop 2023-03-03T06:27:25.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.306+0000 7fa6754fd700 1 -- 172.21.15.3:0/934447842 shutdown_connections 2023-03-03T06:27:25.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.306+0000 7fa6754fd700 1 --2- 172.21.15.3:0/934447842 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6700cb9a0 0x7fa6700cace0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.306+0000 7fa6754fd700 1 -- 172.21.15.3:0/934447842 >> 172.21.15.3:0/934447842 conn(0x7fa6700607c0 msgr2=0x7fa670060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:25.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.306+0000 7fa6754fd700 1 -- 172.21.15.3:0/934447842 shutdown_connections 2023-03-03T06:27:25.312 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.306+0000 7fa6754fd700 1 -- 172.21.15.3:0/934447842 wait complete. 2023-03-03T06:27:25.313 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa6754fd700 1 Processor -- start 2023-03-03T06:27:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa6754fd700 1 -- start start 2023-03-03T06:27:25.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa6754fd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa67006a780 0x7fa6700c3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:25.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa6754fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6700710c0 0x7fa6701f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:25.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa6754fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6700cb9a0 0x7fa6701f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:25.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa6754fd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700c3070 con 0x7fa67006a780 2023-03-03T06:27:25.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa6754fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700c2460 con 0x7fa6700710c0 2023-03-03T06:27:25.316 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa6754fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700c2740 con 0x7fa6700cb9a0 2023-03-03T06:27:25.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa66ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa67006a780 0x7fa6700c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:25.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa674cfc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6700cb9a0 0x7fa6701f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:25.317 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa66ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa67006a780 0x7fa6700c3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52708/0 (socket says 172.21.15.3:52708) 2023-03-03T06:27:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa66ffff700 1 -- 172.21.15.3:0/4141240982 learned_addr learned my addr 172.21.15.3:0/4141240982 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa66f7fe700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6700710c0 0x7fa6701f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa674cfc700 1 -- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6700710c0 msgr2=0x7fa6701f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa674cfc700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6700710c0 0x7fa6701f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.318 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa674cfc700 1 -- 172.21.15.3:0/4141240982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa67006a780 msgr2=0x7fa6700c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.310+0000 7fa674cfc700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa67006a780 0x7fa6700c3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.314+0000 7fa674cfc700 1 -- 172.21.15.3:0/4141240982 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa668009a70 con 0x7fa6700cb9a0 2023-03-03T06:27:25.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.314+0000 7fa66ffff700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa67006a780 0x7fa6700c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:27:25.319 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.314+0000 7fa674cfc700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6700cb9a0 0x7fa6701f75c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa668016000 tx=0x7fa668004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.314+0000 7fa66cff9700 1 -- 172.21.15.3:0/4141240982 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668012430 con 0x7fa6700cb9a0 2023-03-03T06:27:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.314+0000 7fa66cff9700 1 -- 172.21.15.3:0/4141240982 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa668020070 con 0x7fa6700cb9a0 2023-03-03T06:27:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.314+0000 7fa66cff9700 1 -- 172.21.15.3:0/4141240982 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66801ba90 con 0x7fa6700cb9a0 2023-03-03T06:27:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.314+0000 7fa6754fd700 1 -- 172.21.15.3:0/4141240982 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67015c7d0 con 0x7fa6700cb9a0 2023-03-03T06:27:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.314+0000 7fa6754fd700 1 -- 172.21.15.3:0/4141240982 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67015c4f0 con 0x7fa6700cb9a0 2023-03-03T06:27:25.321 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.318+0000 7fa6754fd700 1 -- 172.21.15.3:0/4141240982 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67015cab0 con 0x7fa6700cb9a0 2023-03-03T06:27:25.323 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.318+0000 7fa66cff9700 1 -- 172.21.15.3:0/4141240982 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7fa66801bbf0 con 0x7fa6700cb9a0 2023-03-03T06:27:25.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.318+0000 7fa66cff9700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa65c065000 0x7fa65c0674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:25.325 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.322+0000 7fa66cff9700 1 -- 172.21.15.3:0/4141240982 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7fa668022070 con 0x7fa6700cb9a0 2023-03-03T06:27:25.326 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.322+0000 7fa66ffff700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa65c065000 0x7fa65c0674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:25.327 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.322+0000 7fa66ffff700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa65c065000 0x7fa65c0674b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6600011e0 tx=0x7fa660009040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:25.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.326+0000 7fa66cff9700 1 -- 172.21.15.3:0/4141240982 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa6680525c0 con 0x7fa6700cb9a0 2023-03-03T06:27:25.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.526+0000 7fa6754fd700 1 -- 172.21.15.3:0/4141240982 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6700c2880 con 0x7fa65c065000 2023-03-03T06:27:25.648 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.642+0000 7fa66cff9700 1 -- 172.21.15.3:0/4141240982 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa6700c2880 con 0x7fa65c065000 2023-03-03T06:27:25.648 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-03-03T06:27:25.652 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.646+0000 7fa6754fd700 1 -- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa65c065000 msgr2=0x7fa65c0674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.646+0000 7fa6754fd700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa65c065000 0x7fa65c0674b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6600011e0 tx=0x7fa660009040 comp rx=0 tx=0).stop 2023-03-03T06:27:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.646+0000 7fa6754fd700 1 -- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6700cb9a0 msgr2=0x7fa6701f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.646+0000 7fa6754fd700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6700cb9a0 0x7fa6701f75c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa668016000 tx=0x7fa668004af0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.646+0000 7fa6754fd700 1 -- 172.21.15.3:0/4141240982 shutdown_connections 2023-03-03T06:27:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.646+0000 7fa6754fd700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fa65c065000 0x7fa65c0674b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.646+0000 7fa6754fd700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fa6700cb9a0 0x7fa6701f75c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.653 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.646+0000 7fa6754fd700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6700710c0 0x7fa6701f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.646+0000 7fa6754fd700 1 --2- 172.21.15.3:0/4141240982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa67006a780 0x7fa6700c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.646+0000 7fa6754fd700 1 -- 172.21.15.3:0/4141240982 >> 172.21.15.3:0/4141240982 conn(0x7fa6700607c0 msgr2=0x7fa6701e2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:25.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.650+0000 7fa6754fd700 1 -- 172.21.15.3:0/4141240982 shutdown_connections 2023-03-03T06:27:25.654 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.650+0000 7fa6754fd700 1 -- 172.21.15.3:0/4141240982 wait complete. 2023-03-03T06:27:25.691 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-03-03T06:27:25.691 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir7 1M 2023-03-03T06:27:25.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.910+0000 7f2d24ca4700 1 Processor -- start 2023-03-03T06:27:25.917 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.910+0000 7f2d24ca4700 1 -- start start 2023-03-03T06:27:25.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.914+0000 7f2d24ca4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d200d0b30 0x7f2d200d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:25.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.914+0000 7f2d24ca4700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d200cf3b0 con 0x7f2d2015d6e0 2023-03-03T06:27:25.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.914+0000 7f2d24ca4700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d20156f70 con 0x7f2d20156ba0 2023-03-03T06:27:25.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.914+0000 7f2d24ca4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200d14f0 con 0x7f2d200d0b30 2023-03-03T06:27:25.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1effd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2d2015d6e0 0x7f2d200cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43096/0 (socket says 172.21.15.3:43096) 2023-03-03T06:27:25.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1effd700 1 -- 172.21.15.3:0/1308420678 learned_addr learned my addr 172.21.15.3:0/1308420678 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:25.922 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1ffff700 1 --2- 172.21.15.3:0/1308420678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d200d0b30 0x7f2d200d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:25.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1e7fc700 1 -- 172.21.15.3:0/1308420678 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239553771 0 0) 0x7f2d200cf3b0 con 0x7f2d2015d6e0 2023-03-03T06:27:25.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1e7fc700 1 -- 172.21.15.3:0/1308420678 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d080035d0 con 0x7f2d2015d6e0 2023-03-03T06:27:25.923 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1e7fc700 1 -- 172.21.15.3:0/1308420678 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394836301 0 0) 0x7f2d20156f70 con 0x7f2d20156ba0 2023-03-03T06:27:25.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1e7fc700 1 -- 172.21.15.3:0/1308420678 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d200cf3b0 con 0x7f2d20156ba0 2023-03-03T06:27:25.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1ffff700 1 -- 172.21.15.3:0/1308420678 >> v1:172.21.15.32:6789/0 conn(0x7f2d20156ba0 legacy=0x7f2d20071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1ffff700 1 -- 172.21.15.3:0/1308420678 >> v1:172.21.15.3:6789/0 conn(0x7f2d2015d6e0 legacy=0x7f2d200cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1ffff700 1 -- 172.21.15.3:0/1308420678 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d200cd320 con 0x7f2d200d0b30 2023-03-03T06:27:25.924 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1ffff700 1 --2- 172.21.15.3:0/1308420678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d200d0b30 0x7f2d200d0fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009dc0 tx=0x7f2d1800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c92b44474a8e0ea server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:25.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1e7fc700 1 -- 172.21.15.3:0/1308420678 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800e070 con 0x7f2d200d0b30 2023-03-03T06:27:25.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1e7fc700 1 -- 172.21.15.3:0/1308420678 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1800bb20 con 0x7f2d200d0b30 2023-03-03T06:27:25.925 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.918+0000 7f2d1e7fc700 1 -- 172.21.15.3:0/1308420678 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18012430 con 0x7f2d200d0b30 2023-03-03T06:27:25.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.922+0000 7f2d24ca4700 1 -- 172.21.15.3:0/1308420678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d200d0b30 msgr2=0x7f2d200d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.926 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.922+0000 7f2d24ca4700 1 --2- 172.21.15.3:0/1308420678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d200d0b30 0x7f2d200d0fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009dc0 tx=0x7f2d1800b330 comp rx=0 tx=0).stop 2023-03-03T06:27:25.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.922+0000 7f2d24ca4700 1 -- 172.21.15.3:0/1308420678 shutdown_connections 2023-03-03T06:27:25.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.922+0000 7f2d24ca4700 1 --2- 172.21.15.3:0/1308420678 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d200d0b30 0x7f2d200d0fb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.922+0000 7f2d24ca4700 1 -- 172.21.15.3:0/1308420678 >> 172.21.15.3:0/1308420678 conn(0x7f2d2006e5c0 msgr2=0x7f2d2006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:25.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.922+0000 7f2d24ca4700 1 -- 172.21.15.3:0/1308420678 shutdown_connections 2023-03-03T06:27:25.927 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.922+0000 7f2d24ca4700 1 -- 172.21.15.3:0/1308420678 wait complete. 2023-03-03T06:27:25.928 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.922+0000 7f2d24ca4700 1 Processor -- start 2023-03-03T06:27:25.929 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.922+0000 7f2d24ca4700 1 -- start start 2023-03-03T06:27:25.930 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d24ca4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d200d0b30 0x7f2d20155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:25.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d24ca4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d20156ba0 0x7f2d2015fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:25.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d24ca4700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d2015d6e0 0x7f2d200cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:25.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d24ca4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200c6dd0 con 0x7f2d20156ba0 2023-03-03T06:27:25.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d24ca4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2015def0 con 0x7f2d200d0b30 2023-03-03T06:27:25.931 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d24ca4700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2015e1d0 con 0x7f2d2015d6e0 2023-03-03T06:27:25.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d1effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d20156ba0 0x7f2d2015fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:25.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d1effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d20156ba0 0x7f2d2015fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52718/0 (socket says 172.21.15.3:52718) 2023-03-03T06:27:25.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d1effd700 1 -- 172.21.15.3:0/978443336 learned_addr learned my addr 172.21.15.3:0/978443336 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:25.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d1f7fe700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d200d0b30 0x7f2d20155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:25.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d1ffff700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d2015d6e0 0x7f2d200cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d1effd700 1 -- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d2015d6e0 msgr2=0x7f2d200cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d1effd700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d2015d6e0 0x7f2d200cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d1effd700 1 -- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d200d0b30 msgr2=0x7f2d20155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d1effd700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d200d0b30 0x7f2d20155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d1effd700 1 -- 172.21.15.3:0/978443336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d18009a70 con 0x7f2d20156ba0 2023-03-03T06:27:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d1effd700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d20156ba0 0x7f2d2015fa80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c560 tx=0x7f2d1400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d07fff700 1 -- 172.21.15.3:0/978443336 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14005d90 con 0x7f2d20156ba0 2023-03-03T06:27:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d07fff700 1 -- 172.21.15.3:0/978443336 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d140070d0 con 0x7f2d20156ba0 2023-03-03T06:27:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d07fff700 1 -- 172.21.15.3:0/978443336 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14007440 con 0x7f2d20156ba0 2023-03-03T06:27:25.934 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.926+0000 7f2d24ca4700 1 -- 172.21.15.3:0/978443336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d200cd8e0 con 0x7f2d20156ba0 2023-03-03T06:27:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.930+0000 7f2d24ca4700 1 -- 172.21.15.3:0/978443336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d200cd600 con 0x7f2d20156ba0 2023-03-03T06:27:25.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.930+0000 7f2d24ca4700 1 -- 172.21.15.3:0/978443336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d200cff50 con 0x7f2d20156ba0 2023-03-03T06:27:25.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.934+0000 7f2d07fff700 1 -- 172.21.15.3:0/978443336 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f2d14019030 con 0x7f2d20156ba0 2023-03-03T06:27:25.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.934+0000 7f2d07fff700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2d0c065000 0x7f2d0c0674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:25.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.934+0000 7f2d07fff700 1 -- 172.21.15.3:0/978443336 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f2d14081520 con 0x7f2d20156ba0 2023-03-03T06:27:25.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.934+0000 7f2d1f7fe700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2d0c065000 0x7f2d0c0674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:25.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.938+0000 7f2d1f7fe700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2d0c065000 0x7f2d0c0674b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d100011e0 tx=0x7f2d10009040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:25.946 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:25.942+0000 7f2d07fff700 1 -- 172.21.15.3:0/978443336 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f2d1404a400 con 0x7f2d20156ba0 2023-03-03T06:27:26.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.142+0000 7f2d24ca4700 1 -- 172.21.15.3:0/978443336 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d20073e90 con 0x7f2d0c065000 2023-03-03T06:27:26.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.254+0000 7f2d07fff700 1 -- 172.21.15.3:0/978443336 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f2d20073e90 con 0x7f2d0c065000 2023-03-03T06:27:26.261 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-03-03T06:27:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.258+0000 7f2d24ca4700 1 -- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2d0c065000 msgr2=0x7f2d0c0674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.258+0000 7f2d24ca4700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2d0c065000 0x7f2d0c0674b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d100011e0 tx=0x7f2d10009040 comp rx=0 tx=0).stop 2023-03-03T06:27:26.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.258+0000 7f2d24ca4700 1 -- 172.21.15.3:0/978443336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d20156ba0 msgr2=0x7f2d2015fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:26.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.258+0000 7f2d24ca4700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d20156ba0 0x7f2d2015fa80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c560 tx=0x7f2d1400c870 comp rx=0 tx=0).stop 2023-03-03T06:27:26.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.258+0000 7f2d24ca4700 1 -- 172.21.15.3:0/978443336 shutdown_connections 2023-03-03T06:27:26.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.258+0000 7f2d24ca4700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f2d0c065000 0x7f2d0c0674b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:26.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.258+0000 7f2d24ca4700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f2d2015d6e0 0x7f2d200cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:26.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.258+0000 7f2d24ca4700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d20156ba0 0x7f2d2015fa80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:26.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.258+0000 7f2d24ca4700 1 --2- 172.21.15.3:0/978443336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d200d0b30 0x7f2d20155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:26.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.258+0000 7f2d24ca4700 1 -- 172.21.15.3:0/978443336 >> 172.21.15.3:0/978443336 conn(0x7f2d2006e5c0 msgr2=0x7f2d2015cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:26.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.262+0000 7f2d24ca4700 1 -- 172.21.15.3:0/978443336 shutdown_connections 2023-03-03T06:27:26.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.262+0000 7f2d24ca4700 1 -- 172.21.15.3:0/978443336 wait complete. 2023-03-03T06:27:26.303 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-03-03T06:27:26.304 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir8 1M 2023-03-03T06:27:26.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.518+0000 7f23d55f2700 1 Processor -- start 2023-03-03T06:27:26.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.518+0000 7f23d55f2700 1 -- start start 2023-03-03T06:27:26.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.522+0000 7f23d55f2700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23d00d4040 0x7f23d00d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:26.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.522+0000 7f23d55f2700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d00d32c0 con 0x7f23d015c960 2023-03-03T06:27:26.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.522+0000 7f23d55f2700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d00d26b0 con 0x7f23d00b6050 2023-03-03T06:27:26.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.522+0000 7f23d55f2700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d00d2990 con 0x7f23d00d4040 2023-03-03T06:27:26.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.526+0000 7f23cffff700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23d00d4040 0x7f23d00d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:26.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.526+0000 7f23ceffd700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23d00b6050 0x7f23d00b6420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:58298/0 (socket says 172.21.15.3:58298) 2023-03-03T06:27:26.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.526+0000 7f23ce7fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f23d015c960 0x7f23d015ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43106/0 (socket says 172.21.15.3:43106) 2023-03-03T06:27:26.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.526+0000 7f23ceffd700 1 -- 172.21.15.3:0/2976372533 learned_addr learned my addr 172.21.15.3:0/2976372533 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:27:26.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.526+0000 7f23cdffb700 1 -- 172.21.15.3:0/2976372533 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964423533 0 0) 0x7f23d00d26b0 con 0x7f23d00b6050 2023-03-03T06:27:26.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.526+0000 7f23cffff700 1 -- 172.21.15.3:0/2976372533 >> v1:172.21.15.32:6789/0 conn(0x7f23d00b6050 legacy=0x7f23d00b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:26.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.526+0000 7f23cffff700 1 -- 172.21.15.3:0/2976372533 >> v1:172.21.15.3:6789/0 conn(0x7f23d015c960 legacy=0x7f23d015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:26.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.526+0000 7f23cffff700 1 -- 172.21.15.3:0/2976372533 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d00d2c70 con 0x7f23d00d4040 2023-03-03T06:27:26.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.526+0000 7f23cffff700 1 --2- 172.21.15.3:0/2976372533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d00d4040 0x7f23d00d3960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009d90 tx=0x7f23c800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53aa573c59e28968 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:26.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.526+0000 7f23cdffb700 1 -- 172.21.15.3:0/2976372533 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c800e070 con 0x7f23d00d4040 2023-03-03T06:27:26.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.526+0000 7f23cdffb700 1 -- 172.21.15.3:0/2976372533 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c800bad0 con 0x7f23d00d4040 2023-03-03T06:27:26.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.526+0000 7f23cdffb700 1 -- 172.21.15.3:0/2976372533 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c800be80 con 0x7f23d00d4040 2023-03-03T06:27:26.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.530+0000 7f23d55f2700 1 -- 172.21.15.3:0/2976372533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d00d4040 msgr2=0x7f23d00d3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:26.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.530+0000 7f23d55f2700 1 --2- 172.21.15.3:0/2976372533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d00d4040 0x7f23d00d3960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009d90 tx=0x7f23c800b330 comp rx=0 tx=0).stop 2023-03-03T06:27:26.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.530+0000 7f23d55f2700 1 -- 172.21.15.3:0/2976372533 shutdown_connections 2023-03-03T06:27:26.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.530+0000 7f23d55f2700 1 --2- 172.21.15.3:0/2976372533 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d00d4040 0x7f23d00d3960 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:26.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.530+0000 7f23d55f2700 1 -- 172.21.15.3:0/2976372533 >> 172.21.15.3:0/2976372533 conn(0x7f23d006a9b0 msgr2=0x7f23d00d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:26.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.530+0000 7f23d55f2700 1 -- 172.21.15.3:0/2976372533 shutdown_connections 2023-03-03T06:27:26.535 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.530+0000 7f23d55f2700 1 -- 172.21.15.3:0/2976372533 wait complete. 2023-03-03T06:27:26.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.534+0000 7f23d55f2700 1 Processor -- start 2023-03-03T06:27:26.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.534+0000 7f23d55f2700 1 -- start start 2023-03-03T06:27:26.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.534+0000 7f23d55f2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23d00b6050 0x7f23d00c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:26.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.534+0000 7f23d55f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d00d4040 0x7f23d01556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:26.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.534+0000 7f23d55f2700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d015c960 0x7f23d00b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:26.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.534+0000 7f23d55f2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00c78e0 con 0x7f23d00b6050 2023-03-03T06:27:26.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.534+0000 7f23d55f2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0156130 con 0x7f23d00d4040 2023-03-03T06:27:26.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.534+0000 7f23d55f2700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0156410 con 0x7f23d015c960 2023-03-03T06:27:26.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.534+0000 7f23ceffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23d00b6050 0x7f23d00c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:26.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.534+0000 7f23cffff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d015c960 0x7f23d00b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:26.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.534+0000 7f23ce7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d00d4040 0x7f23d01556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:26.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.534+0000 7f23ceffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23d00b6050 0x7f23d00c73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52722/0 (socket says 172.21.15.3:52722) 2023-03-03T06:27:26.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.534+0000 7f23ceffd700 1 -- 172.21.15.3:0/3083426468 learned_addr learned my addr 172.21.15.3:0/3083426468 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:26.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.538+0000 7f23ce7fc700 1 -- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d015c960 msgr2=0x7f23d00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:26.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.538+0000 7f23ce7fc700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d015c960 0x7f23d00b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:26.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.538+0000 7f23ce7fc700 1 -- 172.21.15.3:0/3083426468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23d00b6050 msgr2=0x7f23d00c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:26.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.538+0000 7f23ce7fc700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23d00b6050 0x7f23d00c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:26.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.538+0000 7f23ce7fc700 1 -- 172.21.15.3:0/3083426468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c8009a70 con 0x7f23d00d4040 2023-03-03T06:27:26.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.538+0000 7f23ceffd700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23d00b6050 0x7f23d00c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:27:26.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.538+0000 7f23cffff700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d015c960 0x7f23d00b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:27:26.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.538+0000 7f23ce7fc700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d00d4040 0x7f23d01556e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f23c400c520 tx=0x7f23c400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:26.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.538+0000 7f23b77fe700 1 -- 172.21.15.3:0/3083426468 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4004ca0 con 0x7f23d00d4040 2023-03-03T06:27:26.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.538+0000 7f23b77fe700 1 -- 172.21.15.3:0/3083426468 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c4004e00 con 0x7f23d00d4040 2023-03-03T06:27:26.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.538+0000 7f23d55f2700 1 -- 172.21.15.3:0/3083426468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d00cf7f0 con 0x7f23d00d4040 2023-03-03T06:27:26.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.538+0000 7f23b77fe700 1 -- 172.21.15.3:0/3083426468 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4007310 con 0x7f23d00d4040 2023-03-03T06:27:26.545 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.538+0000 7f23d55f2700 1 -- 172.21.15.3:0/3083426468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d00d2f50 con 0x7f23d00d4040 2023-03-03T06:27:26.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.542+0000 7f23d55f2700 1 -- 172.21.15.3:0/3083426468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d015d7f0 con 0x7f23d00d4040 2023-03-03T06:27:26.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.542+0000 7f23b77fe700 1 -- 172.21.15.3:0/3083426468 <== mon.1 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f23c401a020 con 0x7f23d00d4040 2023-03-03T06:27:26.549 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.542+0000 7f23b77fe700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f23bc0650d0 0x7f23bc067580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:26.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.546+0000 7f23b77fe700 1 -- 172.21.15.3:0/3083426468 <== mon.1 v2:172.21.15.32:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f23c4081da0 con 0x7f23d00d4040 2023-03-03T06:27:26.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.546+0000 7f23ceffd700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f23bc0650d0 0x7f23bc067580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:26.552 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.546+0000 7f23ceffd700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f23bc0650d0 0x7f23bc067580 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f23c0000f60 tx=0x7f23c0007ec0 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:26.558 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.554+0000 7f23b77fe700 1 -- 172.21.15.3:0/3083426468 <== mon.1 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f23c404fbe0 con 0x7f23d00d4040 2023-03-03T06:27:26.759 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.754+0000 7f23d55f2700 1 -- 172.21.15.3:0/3083426468 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f23d004e180 con 0x7f23bc0650d0 2023-03-03T06:27:26.870 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.866+0000 7f23b77fe700 1 -- 172.21.15.3:0/3083426468 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f23d004e180 con 0x7f23bc0650d0 2023-03-03T06:27:26.870 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-03-03T06:27:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.870+0000 7f23d55f2700 1 -- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f23bc0650d0 msgr2=0x7f23bc067580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.870+0000 7f23d55f2700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f23bc0650d0 0x7f23bc067580 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f23c0000f60 tx=0x7f23c0007ec0 comp rx=0 tx=0).stop 2023-03-03T06:27:26.876 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.870+0000 7f23d55f2700 1 -- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d00d4040 msgr2=0x7f23d01556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.870+0000 7f23d55f2700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d00d4040 0x7f23d01556e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f23c400c520 tx=0x7f23c400c830 comp rx=0 tx=0).stop 2023-03-03T06:27:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.870+0000 7f23d55f2700 1 -- 172.21.15.3:0/3083426468 shutdown_connections 2023-03-03T06:27:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.870+0000 7f23d55f2700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f23bc0650d0 0x7f23bc067580 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.870+0000 7f23d55f2700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f23d015c960 0x7f23d00b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.870+0000 7f23d55f2700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d00d4040 0x7f23d01556e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.870+0000 7f23d55f2700 1 --2- 172.21.15.3:0/3083426468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23d00b6050 0x7f23d00c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.870+0000 7f23d55f2700 1 -- 172.21.15.3:0/3083426468 >> 172.21.15.3:0/3083426468 conn(0x7f23d006a9b0 msgr2=0x7f23d00d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.870+0000 7f23d55f2700 1 -- 172.21.15.3:0/3083426468 shutdown_connections 2023-03-03T06:27:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:26.870+0000 7f23d55f2700 1 -- 172.21.15.3:0/3083426468 wait complete. 2023-03-03T06:27:26.912 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-03-03T06:27:26.913 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir9 1M 2023-03-03T06:27:27.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.130+0000 7f765988c700 1 Processor -- start 2023-03-03T06:27:27.136 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.130+0000 7f765988c700 1 -- start start 2023-03-03T06:27:27.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f765988c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f765415d720 0x7f76541608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:27.138 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f765988c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76540cf610 0x7f76540cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:27.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f765988c700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76540cffd0 con 0x7f7654156cc0 2023-03-03T06:27:27.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f765988c700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76541600f0 con 0x7f76540cf610 2023-03-03T06:27:27.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f765988c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76540cd420 con 0x7f765415d720 2023-03-03T06:27:27.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f765908b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76540cf610 0x7f76540cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:27.139 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f7653fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f765415d720 0x7f76541608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:27.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f765908b700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76540cf610 0x7f76540cfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:37006/0 (socket says 172.21.15.3:37006) 2023-03-03T06:27:27.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f765908b700 1 -- 172.21.15.3:0/3721157988 learned_addr learned my addr 172.21.15.3:0/3721157988 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:27.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f765908b700 1 -- 172.21.15.3:0/3721157988 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f765415d720 msgr2=0x7f76541608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:27:27.140 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f765908b700 1 --2- 172.21.15.3:0/3721157988 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f765415d720 0x7f76541608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f765908b700 1 -- 172.21.15.3:0/3721157988 >> v1:172.21.15.32:6789/0 conn(0x7f7654156cc0 legacy=0x7f76540713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f765908b700 1 -- 172.21.15.3:0/3721157988 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76540cd700 con 0x7f76540cf610 2023-03-03T06:27:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f765908b700 1 --2- 172.21.15.3:0/3721157988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76540cf610 0x7f76540cfa90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f764c009e50 tx=0x7f764c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6d6c2edfffaf33 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f76537fe700 1 -- 172.21.15.3:0/3721157988 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c00e070 con 0x7f76540cf610 2023-03-03T06:27:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f76537fe700 1 -- 172.21.15.3:0/3721157988 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c00b910 con 0x7f76540cf610 2023-03-03T06:27:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.134+0000 7f76537fe700 1 -- 172.21.15.3:0/3721157988 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c00bcc0 con 0x7f76540cf610 2023-03-03T06:27:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.138+0000 7f765988c700 1 -- 172.21.15.3:0/3721157988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76540cf610 msgr2=0x7f76540cfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.138+0000 7f765988c700 1 --2- 172.21.15.3:0/3721157988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76540cf610 0x7f76540cfa90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f764c009e50 tx=0x7f764c00b330 comp rx=0 tx=0).stop 2023-03-03T06:27:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.138+0000 7f765988c700 1 -- 172.21.15.3:0/3721157988 shutdown_connections 2023-03-03T06:27:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.138+0000 7f765988c700 1 --2- 172.21.15.3:0/3721157988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76540cf610 0x7f76540cfa90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.138+0000 7f765988c700 1 --2- 172.21.15.3:0/3721157988 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f765415d720 0x7f76541608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.138+0000 7f765988c700 1 -- 172.21.15.3:0/3721157988 >> 172.21.15.3:0/3721157988 conn(0x7f765406e5c0 msgr2=0x7f765406e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.138+0000 7f765988c700 1 -- 172.21.15.3:0/3721157988 shutdown_connections 2023-03-03T06:27:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.138+0000 7f765988c700 1 -- 172.21.15.3:0/3721157988 wait complete. 2023-03-03T06:27:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.138+0000 7f765988c700 1 Processor -- start 2023-03-03T06:27:27.146 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f765988c700 1 -- start start 2023-03-03T06:27:27.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f765988c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76540cf610 0x7f76540cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:27.147 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f765988c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7654156cc0 0x7f76540b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f765988c700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f765415d720 0x7f76540b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f765988c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76540d2890 con 0x7f7654156cc0 2023-03-03T06:27:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f765988c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76540c71e0 con 0x7f76540cf610 2023-03-03T06:27:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f765988c700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76540d2b00 con 0x7f765415d720 2023-03-03T06:27:27.149 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f7653fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7654156cc0 0x7f76540b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:27.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f7653fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7654156cc0 0x7f76540b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52728/0 (socket says 172.21.15.3:52728) 2023-03-03T06:27:27.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f7653fff700 1 -- 172.21.15.3:0/2040433626 learned_addr learned my addr 172.21.15.3:0/2040433626 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:27:27.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f765888a700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76540cf610 0x7f76540cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:27.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f765908b700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f765415d720 0x7f76540b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:27.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f7653fff700 1 -- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f765415d720 msgr2=0x7f76540b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:27.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f7653fff700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f765415d720 0x7f76540b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:27.150 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f7653fff700 1 -- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76540cf610 msgr2=0x7f76540cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:27.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f7653fff700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76540cf610 0x7f76540cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:27.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.142+0000 7f7653fff700 1 -- 172.21.15.3:0/2040433626 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c009b30 con 0x7f7654156cc0 2023-03-03T06:27:27.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.146+0000 7f7653fff700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7654156cc0 0x7f76540b3db0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f764000e9a0 tx=0x7f764000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:27.151 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.146+0000 7f76517fa700 1 -- 172.21.15.3:0/2040433626 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640009720 con 0x7f7654156cc0 2023-03-03T06:27:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.146+0000 7f76517fa700 1 -- 172.21.15.3:0/2040433626 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764000c690 con 0x7f7654156cc0 2023-03-03T06:27:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.146+0000 7f76517fa700 1 -- 172.21.15.3:0/2040433626 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764000ca40 con 0x7f7654156cc0 2023-03-03T06:27:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.146+0000 7f765988c700 1 -- 172.21.15.3:0/2040433626 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76540cd9e0 con 0x7f7654156cc0 2023-03-03T06:27:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.146+0000 7f765988c700 1 -- 172.21.15.3:0/2040433626 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7654160390 con 0x7f7654156cc0 2023-03-03T06:27:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.146+0000 7f765988c700 1 -- 172.21.15.3:0/2040433626 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7654160df0 con 0x7f7654156cc0 2023-03-03T06:27:27.154 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.150+0000 7f76517fa700 1 -- 172.21.15.3:0/2040433626 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f7640019030 con 0x7f7654156cc0 2023-03-03T06:27:27.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.150+0000 7f76517fa700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7638065030 0x7f76380674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:27:27.156 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.150+0000 7f76517fa700 1 -- 172.21.15.3:0/2040433626 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6640+0+0 (secure 0 0 0) 0x7f7640082140 con 0x7f7654156cc0 2023-03-03T06:27:27.157 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.150+0000 7f765888a700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7638065030 0x7f76380674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:27:27.158 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.154+0000 7f765888a700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7638065030 0x7f76380674e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7648000f60 tx=0x7f7648009040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:27:27.163 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.158+0000 7f76517fa700 1 -- 172.21.15.3:0/2040433626 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f764004ff00 con 0x7f7654156cc0 2023-03-03T06:27:27.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.362+0000 7f765988c700 1 -- 172.21.15.3:0/2040433626 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f76540d2c70 con 0x7f7638065030 2023-03-03T06:27:27.476 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.470+0000 7f76517fa700 1 -- 172.21.15.3:0/2040433626 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f76540d2c70 con 0x7f7638065030 2023-03-03T06:27:27.477 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-03-03T06:27:27.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.478+0000 7f765988c700 1 -- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7638065030 msgr2=0x7f76380674e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:27.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.478+0000 7f765988c700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7638065030 0x7f76380674e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7648000f60 tx=0x7f7648009040 comp rx=0 tx=0).stop 2023-03-03T06:27:27.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.478+0000 7f765988c700 1 -- 172.21.15.3:0/2040433626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7654156cc0 msgr2=0x7f76540b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:27:27.482 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.478+0000 7f765988c700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7654156cc0 0x7f76540b3db0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f764000e9a0 tx=0x7f764000ecb0 comp rx=0 tx=0).stop 2023-03-03T06:27:27.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.478+0000 7f765988c700 1 -- 172.21.15.3:0/2040433626 shutdown_connections 2023-03-03T06:27:27.483 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.478+0000 7f765988c700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7638065030 0x7f76380674e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:27.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.478+0000 7f765988c700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f765415d720 0x7f76540b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:27.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.478+0000 7f765988c700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7654156cc0 0x7f76540b3db0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:27.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.478+0000 7f765988c700 1 --2- 172.21.15.3:0/2040433626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76540cf610 0x7f76540cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:27:27.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.478+0000 7f765988c700 1 -- 172.21.15.3:0/2040433626 >> 172.21.15.3:0/2040433626 conn(0x7f765406e5c0 msgr2=0x7f76540713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:27:27.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.478+0000 7f765988c700 1 -- 172.21.15.3:0/2040433626 shutdown_connections 2023-03-03T06:27:27.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:27:27.478+0000 7f765988c700 1 -- 172.21.15.3:0/2040433626 wait complete. 2023-03-03T06:27:27.519 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-03-03T06:27:27.557 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:27:27.565 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:27:57.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:27:57.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:28:00.040 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:28:00.034+0000 7f448fd18700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:28:00.041 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:28:00.034+0000 7f448fd18700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:28:00.041 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:28:00.034+0000 7f448fd18700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:28:00.041 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:28:00.034+0000 7f448fd18700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:28:00.041 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:28:00.034+0000 7f448fd18700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:28:27.728 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:28:27.737 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:28:57.816 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:28:57.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:29:27.900 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:29:27.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:29:57.987 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:29:57.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:30:28.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:30:28.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:30:58.162 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:30:58.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:31:28.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:31:28.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:31:58.366 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:31:58.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:32:28.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:32:28.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:32:58.563 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:32:58.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:33:00.457 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:33:00.452+0000 7f4490d1a700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:33:00.458 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:33:00.452+0000 7f4490d1a700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:33:00.458 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:33:00.452+0000 7f4490d1a700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:33:00.458 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:33:00.452+0000 7f4490d1a700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:33:00.458 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:33:00.452+0000 7f4490d1a700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:33:28.652 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:33:28.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:33:58.740 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:33:58.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:34:28.822 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:34:28.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:34:58.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:34:58.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:35:00.886 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:35:00.881+0000 7f448fd18700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:35:00.887 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:35:00.881+0000 7f448fd18700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:35:00.887 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:35:00.881+0000 7f448fd18700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:35:00.887 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:35:00.881+0000 7f448fd18700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:35:00.887 INFO:tasks.ceph.mgr.x.smithi032.stderr:2023-03-03T06:35:00.881+0000 7f448fd18700 -1 client.0 error registering admin socket command: (17) File exists 2023-03-03T06:35:29.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:35:29.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:35:59.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:35:59.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:36:29.182 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:36:29.192 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:36:59.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:36:59.280 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:37:29.379 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:37:29.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:37:59.475 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:37:59.483 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:38:21.520 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir0 1M 2023-03-03T06:38:21.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.729+0000 7f5b11a4a700 1 Processor -- start 2023-03-03T06:38:21.739 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.729+0000 7f5b11a4a700 1 -- start start 2023-03-03T06:38:21.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b11a4a700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b0c0b1c40 0x7f5b0c15d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:21.743 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b11a4a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b0c15d9f0 0x7f5b0c0cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:21.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b11a4a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0c15ccd0 con 0x7f5b0c1592a0 2023-03-03T06:38:21.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b0bfff700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b0c15d9f0 0x7f5b0c0cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:21.744 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b0affd700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5b0c1592a0 0x7f5b0c159fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36796/0 (socket says 172.21.15.3:36796) 2023-03-03T06:38:21.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b0affd700 1 -- 172.21.15.3:0/3633391719 learned_addr learned my addr 172.21.15.3:0/3633391719 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:38:21.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b11a4a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b0c0b2070 con 0x7f5b0c15d9f0 2023-03-03T06:38:21.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b11a4a700 1 -- 172.21.15.3:0/3633391719 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b0c0b2290 con 0x7f5b0c0b1c40 2023-03-03T06:38:21.745 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b0a7fc700 1 --2- 172.21.15.3:0/3633391719 >> v2:172.21.15.32:3300/0 conn(0x7f5b0c0b1c40 0x7f5b0c15d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b09ffb700 1 -- 172.21.15.3:0/3633391719 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13520787 0 0) 0x7f5b0c15ccd0 con 0x7f5b0c1592a0 2023-03-03T06:38:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b09ffb700 1 -- 172.21.15.3:0/3633391719 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af40035d0 con 0x7f5b0c1592a0 2023-03-03T06:38:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b0a7fc700 1 -- 172.21.15.3:0/3633391719 >> v2:172.21.15.3:3300/0 conn(0x7f5b0c15d9f0 msgr2=0x7f5b0c0cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b0a7fc700 1 --2- 172.21.15.3:0/3633391719 >> v2:172.21.15.3:3300/0 conn(0x7f5b0c15d9f0 0x7f5b0c0cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:21.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b0a7fc700 1 -- 172.21.15.3:0/3633391719 >> v1:172.21.15.3:6789/0 conn(0x7f5b0c1592a0 legacy=0x7f5b0c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:21.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b0a7fc700 1 -- 172.21.15.3:0/3633391719 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0b2570 con 0x7f5b0c0b1c40 2023-03-03T06:38:21.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.733+0000 7f5b0bfff700 1 --2- 172.21.15.3:0/3633391719 >> v2:172.21.15.3:3300/0 conn(0x7f5b0c15d9f0 0x7f5b0c0cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:38:21.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.737+0000 7f5b0a7fc700 1 --2- 172.21.15.3:0/3633391719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0c0b1c40 0x7f5b0c15d4b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0cc730 tx=0x7f5b00009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=135eb72e111dd2d1 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:21.747 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.737+0000 7f5b09ffb700 1 -- 172.21.15.3:0/3633391719 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00011830 con 0x7f5b0c0b1c40 2023-03-03T06:38:21.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.737+0000 7f5b09ffb700 1 -- 172.21.15.3:0/3633391719 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b00011990 con 0x7f5b0c0b1c40 2023-03-03T06:38:21.748 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.737+0000 7f5b09ffb700 1 -- 172.21.15.3:0/3633391719 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00011d00 con 0x7f5b0c0b1c40 2023-03-03T06:38:21.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.737+0000 7f5b11a4a700 1 -- 172.21.15.3:0/3633391719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0c0b1c40 msgr2=0x7f5b0c15d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:21.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.737+0000 7f5b11a4a700 1 --2- 172.21.15.3:0/3633391719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0c0b1c40 0x7f5b0c15d4b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0cc730 tx=0x7f5b00009b30 comp rx=0 tx=0).stop 2023-03-03T06:38:21.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.737+0000 7f5b11a4a700 1 -- 172.21.15.3:0/3633391719 shutdown_connections 2023-03-03T06:38:21.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.737+0000 7f5b11a4a700 1 --2- 172.21.15.3:0/3633391719 >> v2:172.21.15.3:3300/0 conn(0x7f5b0c15d9f0 0x7f5b0c0cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:21.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.737+0000 7f5b11a4a700 1 --2- 172.21.15.3:0/3633391719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0c0b1c40 0x7f5b0c15d4b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:21.749 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.737+0000 7f5b11a4a700 1 -- 172.21.15.3:0/3633391719 >> 172.21.15.3:0/3633391719 conn(0x7f5b0c0ccc40 msgr2=0x7f5b0c0cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:21.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.737+0000 7f5b11a4a700 1 -- 172.21.15.3:0/3633391719 shutdown_connections 2023-03-03T06:38:21.750 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.737+0000 7f5b11a4a700 1 -- 172.21.15.3:0/3633391719 wait complete. 2023-03-03T06:38:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.741+0000 7f5b11a4a700 1 Processor -- start 2023-03-03T06:38:21.751 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.741+0000 7f5b11a4a700 1 -- start start 2023-03-03T06:38:21.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.741+0000 7f5b11a4a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0c0b1c40 0x7f5b0c0c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:21.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.741+0000 7f5b11a4a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b0c1592a0 0x7f5b0c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:21.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.741+0000 7f5b11a4a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5b0c15d9f0 0x7f5b0c145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:21.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.741+0000 7f5b11a4a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0b3f30 con 0x7f5b0c1592a0 2023-03-03T06:38:21.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.741+0000 7f5b11a4a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0b43c0 con 0x7f5b0c0b1c40 2023-03-03T06:38:21.753 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.741+0000 7f5b11a4a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0b5be0 con 0x7f5b0c15d9f0 2023-03-03T06:38:21.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.741+0000 7f5b0a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b0c1592a0 0x7f5b0c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:21.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.741+0000 7f5b0a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b0c1592a0 0x7f5b0c0b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50984/0 (socket says 172.21.15.3:50984) 2023-03-03T06:38:21.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.741+0000 7f5b0a7fc700 1 -- 172.21.15.3:0/2381862841 learned_addr learned my addr 172.21.15.3:0/2381862841 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:21.754 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5b0affd700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0c0b1c40 0x7f5b0c0c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:21.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5b0bfff700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5b0c15d9f0 0x7f5b0c145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:21.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5b0a7fc700 1 -- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5b0c15d9f0 msgr2=0x7f5b0c145360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:21.755 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5b0a7fc700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5b0c15d9f0 0x7f5b0c145360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5b0a7fc700 1 -- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0c0b1c40 msgr2=0x7f5b0c0c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5b0a7fc700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0c0b1c40 0x7f5b0c0c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5b0a7fc700 1 -- 172.21.15.3:0/2381862841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b04009a70 con 0x7f5b0c1592a0 2023-03-03T06:38:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5b0bfff700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5b0c15d9f0 0x7f5b0c145360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:38:21.756 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5b0a7fc700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b0c1592a0 0x7f5b0c0b3260 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009b70 tx=0x7f5b00004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5af3fff700 1 -- 172.21.15.3:0/2381862841 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00004550 con 0x7f5b0c1592a0 2023-03-03T06:38:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5af3fff700 1 -- 172.21.15.3:0/2381862841 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0001f070 con 0x7f5b0c1592a0 2023-03-03T06:38:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5af3fff700 1 -- 172.21.15.3:0/2381862841 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b000038c0 con 0x7f5b0c1592a0 2023-03-03T06:38:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5b11a4a700 1 -- 172.21.15.3:0/2381862841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b000094b0 con 0x7f5b0c1592a0 2023-03-03T06:38:21.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.745+0000 7f5b11a4a700 1 -- 172.21.15.3:0/2381862841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c15cf90 con 0x7f5b0c1592a0 2023-03-03T06:38:21.758 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.749+0000 7f5b11a4a700 1 -- 172.21.15.3:0/2381862841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0c0b2850 con 0x7f5b0c1592a0 2023-03-03T06:38:21.760 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.749+0000 7f5af3fff700 1 -- 172.21.15.3:0/2381862841 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f5b0000f050 con 0x7f5b0c1592a0 2023-03-03T06:38:21.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.753+0000 7f5af3fff700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5af8065000 0x7f5af80674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:21.762 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.753+0000 7f5af3fff700 1 -- 172.21.15.3:0/2381862841 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f5b00021070 con 0x7f5b0c1592a0 2023-03-03T06:38:21.763 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.753+0000 7f5b0affd700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5af8065000 0x7f5af80674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:21.764 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.753+0000 7f5b0affd700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5af8065000 0x7f5af80674b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5afc004cb0 tx=0x7f5afc007d60 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:21.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.757+0000 7f5af3fff700 1 -- 172.21.15.3:0/2381862841 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5b00051940 con 0x7f5b0c1592a0 2023-03-03T06:38:21.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:21.961+0000 7f5b11a4a700 1 -- 172.21.15.3:0/2381862841 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b0c06ee80 con 0x7f5af8065000 2023-03-03T06:38:22.129 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.117+0000 7f5af3fff700 1 -- 172.21.15.3:0/2381862841 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f5b0c06ee80 con 0x7f5af8065000 2023-03-03T06:38:22.129 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-03-03T06:38:22.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.121+0000 7f5b11a4a700 1 -- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5af8065000 msgr2=0x7f5af80674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:22.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.121+0000 7f5b11a4a700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5af8065000 0x7f5af80674b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5afc004cb0 tx=0x7f5afc007d60 comp rx=0 tx=0).stop 2023-03-03T06:38:22.133 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.121+0000 7f5b11a4a700 1 -- 172.21.15.3:0/2381862841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b0c1592a0 msgr2=0x7f5b0c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:22.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.121+0000 7f5b11a4a700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b0c1592a0 0x7f5b0c0b3260 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009b70 tx=0x7f5b00004770 comp rx=0 tx=0).stop 2023-03-03T06:38:22.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.121+0000 7f5b11a4a700 1 -- 172.21.15.3:0/2381862841 shutdown_connections 2023-03-03T06:38:22.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.121+0000 7f5b11a4a700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5af8065000 0x7f5af80674b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:22.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.121+0000 7f5b11a4a700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5b0c15d9f0 0x7f5b0c145360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:22.134 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.121+0000 7f5b11a4a700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b0c1592a0 0x7f5b0c0b3260 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:22.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.121+0000 7f5b11a4a700 1 --2- 172.21.15.3:0/2381862841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b0c0b1c40 0x7f5b0c0c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:22.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.121+0000 7f5b11a4a700 1 -- 172.21.15.3:0/2381862841 >> 172.21.15.3:0/2381862841 conn(0x7f5b0c0ccc40 msgr2=0x7f5b0c06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:22.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.125+0000 7f5b11a4a700 1 -- 172.21.15.3:0/2381862841 shutdown_connections 2023-03-03T06:38:22.135 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.125+0000 7f5b11a4a700 1 -- 172.21.15.3:0/2381862841 wait complete. 2023-03-03T06:38:22.169 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-03-03T06:38:22.170 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir1 1M 2023-03-03T06:38:22.388 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.377+0000 7f48b759e700 1 Processor -- start 2023-03-03T06:38:22.389 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.377+0000 7f48b759e700 1 -- start start 2023-03-03T06:38:22.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48b8156ba0 0x7f48b8071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:22.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48b815d6e0 0x7f48b80cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:22.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b80cf3b0 con 0x7f48b80cfd80 2023-03-03T06:38:22.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b8156f70 con 0x7f48b8156ba0 2023-03-03T06:38:22.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b759e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80cd320 con 0x7f48b815d6e0 2023-03-03T06:38:22.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48b8156ba0 0x7f48b8071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:22.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b659c700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f48b8156ba0 0x7f48b8071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:57644/0 (socket says 172.21.15.3:57644) 2023-03-03T06:38:22.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b659c700 1 -- 172.21.15.3:0/2823135767 learned_addr learned my addr 172.21.15.3:0/2823135767 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:22.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b6d9d700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f48b80cfd80 0x7f48b80d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:60630/0 (socket says 172.21.15.3:60630) 2023-03-03T06:38:22.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48affff700 1 --2- 172.21.15.3:0/2823135767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48b815d6e0 0x7f48b80cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:22.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b659c700 1 -- 172.21.15.3:0/2823135767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48b815d6e0 msgr2=0x7f48b80cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:22.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b659c700 1 --2- 172.21.15.3:0/2823135767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48b815d6e0 0x7f48b80cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:22.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b659c700 1 -- 172.21.15.3:0/2823135767 >> v1:172.21.15.32:6789/0 conn(0x7f48b80cfd80 legacy=0x7f48b80d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:22.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b659c700 1 -- 172.21.15.3:0/2823135767 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b80cd600 con 0x7f48b8156ba0 2023-03-03T06:38:22.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48affff700 1 --2- 172.21.15.3:0/2823135767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48b815d6e0 0x7f48b80cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:38:22.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.381+0000 7f48b659c700 1 --2- 172.21.15.3:0/2823135767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b8156ba0 0x7f48b8071280 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009910 tx=0x7f48a8009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e5567b518c24c25 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:22.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.385+0000 7f48b5d9b700 1 -- 172.21.15.3:0/2823135767 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a800ba60 con 0x7f48b8156ba0 2023-03-03T06:38:22.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.385+0000 7f48b5d9b700 1 -- 172.21.15.3:0/2823135767 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a800bbc0 con 0x7f48b8156ba0 2023-03-03T06:38:22.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.385+0000 7f48b5d9b700 1 -- 172.21.15.3:0/2823135767 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a8009e60 con 0x7f48b8156ba0 2023-03-03T06:38:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.385+0000 7f48b759e700 1 -- 172.21.15.3:0/2823135767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b8156ba0 msgr2=0x7f48b8071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.385+0000 7f48b759e700 1 --2- 172.21.15.3:0/2823135767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b8156ba0 0x7f48b8071280 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009910 tx=0x7f48a8009c20 comp rx=0 tx=0).stop 2023-03-03T06:38:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.385+0000 7f48b759e700 1 -- 172.21.15.3:0/2823135767 shutdown_connections 2023-03-03T06:38:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.385+0000 7f48b759e700 1 --2- 172.21.15.3:0/2823135767 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48b815d6e0 0x7f48b80cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.385+0000 7f48b759e700 1 --2- 172.21.15.3:0/2823135767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b8156ba0 0x7f48b8071280 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:22.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.385+0000 7f48b759e700 1 -- 172.21.15.3:0/2823135767 >> 172.21.15.3:0/2823135767 conn(0x7f48b806e5c0 msgr2=0x7f48b806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:22.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.385+0000 7f48b759e700 1 -- 172.21.15.3:0/2823135767 shutdown_connections 2023-03-03T06:38:22.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.385+0000 7f48b759e700 1 -- 172.21.15.3:0/2823135767 wait complete. 2023-03-03T06:38:22.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.389+0000 7f48b759e700 1 Processor -- start 2023-03-03T06:38:22.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.389+0000 7f48b759e700 1 -- start start 2023-03-03T06:38:22.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.389+0000 7f48b759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b80cfd80 0x7f48b80b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:22.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.389+0000 7f48b759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48b8156ba0 0x7f48b80b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:22.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.389+0000 7f48b759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b815d6e0 0x7f48b80b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:22.402 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.389+0000 7f48b759e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b81f5d60 con 0x7f48b80cfd80 2023-03-03T06:38:22.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.389+0000 7f48b759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80c9d00 con 0x7f48b815d6e0 2023-03-03T06:38:22.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.389+0000 7f48b759e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80c9f50 con 0x7f48b8156ba0 2023-03-03T06:38:22.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48b659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b80cfd80 0x7f48b80b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:22.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48b659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b80cfd80 0x7f48b80b5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50994/0 (socket says 172.21.15.3:50994) 2023-03-03T06:38:22.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48b659c700 1 -- 172.21.15.3:0/3920819251 learned_addr learned my addr 172.21.15.3:0/3920819251 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:22.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48affff700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48b8156ba0 0x7f48b80b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:22.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48b6d9d700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b815d6e0 0x7f48b80b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:22.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48b659c700 1 -- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48b8156ba0 msgr2=0x7f48b80b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:22.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48b659c700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48b8156ba0 0x7f48b80b5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:22.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48b659c700 1 -- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b815d6e0 msgr2=0x7f48b80b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:22.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48b659c700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b815d6e0 0x7f48b80b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:22.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48b659c700 1 -- 172.21.15.3:0/3920819251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a8009570 con 0x7f48b80cfd80 2023-03-03T06:38:22.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48b659c700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b80cfd80 0x7f48b80b5a60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f48a800b3c0 tx=0x7f48a8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:22.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48aeffd700 1 -- 172.21.15.3:0/3920819251 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a8004590 con 0x7f48b80cfd80 2023-03-03T06:38:22.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48aeffd700 1 -- 172.21.15.3:0/3920819251 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a801f070 con 0x7f48b80cfd80 2023-03-03T06:38:22.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48aeffd700 1 -- 172.21.15.3:0/3920819251 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a8003900 con 0x7f48b80cfd80 2023-03-03T06:38:22.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48b759e700 1 -- 172.21.15.3:0/3920819251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b80717c0 con 0x7f48b80cfd80 2023-03-03T06:38:22.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.393+0000 7f48b759e700 1 -- 172.21.15.3:0/3920819251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b80cd8e0 con 0x7f48b80cfd80 2023-03-03T06:38:22.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.397+0000 7f48b759e700 1 -- 172.21.15.3:0/3920819251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b80cef40 con 0x7f48b80cfd80 2023-03-03T06:38:22.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.397+0000 7f48aeffd700 1 -- 172.21.15.3:0/3920819251 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f48a800f050 con 0x7f48b80cfd80 2023-03-03T06:38:22.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.401+0000 7f48aeffd700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4898065000 0x7f48980674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:22.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.401+0000 7f48aeffd700 1 -- 172.21.15.3:0/3920819251 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f48a8021070 con 0x7f48b80cfd80 2023-03-03T06:38:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.401+0000 7f48affff700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4898065000 0x7f48980674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:22.414 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.405+0000 7f48affff700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4898065000 0x7f48980674b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f48b806e150 tx=0x7f48a0008040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:22.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.409+0000 7f48aeffd700 1 -- 172.21.15.3:0/3920819251 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f48a80518c0 con 0x7f48b80cfd80 2023-03-03T06:38:22.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.605+0000 7f48b759e700 1 -- 172.21.15.3:0/3920819251 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f48b8073e90 con 0x7f4898065000 2023-03-03T06:38:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.765+0000 7f48aeffd700 1 -- 172.21.15.3:0/3920819251 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f48b8073e90 con 0x7f4898065000 2023-03-03T06:38:22.778 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-03-03T06:38:22.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.773+0000 7f48b759e700 1 -- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4898065000 msgr2=0x7f48980674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:22.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.773+0000 7f48b759e700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4898065000 0x7f48980674b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f48b806e150 tx=0x7f48a0008040 comp rx=0 tx=0).stop 2023-03-03T06:38:22.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.773+0000 7f48b759e700 1 -- 172.21.15.3:0/3920819251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b80cfd80 msgr2=0x7f48b80b5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:22.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.773+0000 7f48b759e700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b80cfd80 0x7f48b80b5a60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f48a800b3c0 tx=0x7f48a8004770 comp rx=0 tx=0).stop 2023-03-03T06:38:22.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.773+0000 7f48b759e700 1 -- 172.21.15.3:0/3920819251 shutdown_connections 2023-03-03T06:38:22.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.773+0000 7f48b759e700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4898065000 0x7f48980674b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:22.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.773+0000 7f48b759e700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48b815d6e0 0x7f48b80b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:22.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.773+0000 7f48b759e700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f48b8156ba0 0x7f48b80b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:22.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.773+0000 7f48b759e700 1 --2- 172.21.15.3:0/3920819251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48b80cfd80 0x7f48b80b5a60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:22.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.773+0000 7f48b759e700 1 -- 172.21.15.3:0/3920819251 >> 172.21.15.3:0/3920819251 conn(0x7f48b806e5c0 msgr2=0x7f48b815e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:22.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.773+0000 7f48b759e700 1 -- 172.21.15.3:0/3920819251 shutdown_connections 2023-03-03T06:38:22.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:22.773+0000 7f48b759e700 1 -- 172.21.15.3:0/3920819251 wait complete. 2023-03-03T06:38:22.818 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-03-03T06:38:22.818 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir2 1M 2023-03-03T06:38:23.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.025+0000 7efe65551700 1 Processor -- start 2023-03-03T06:38:23.036 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.025+0000 7efe65551700 1 -- start start 2023-03-03T06:38:23.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe65551700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe600ce150 0x7efe600ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe65551700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe60070020 con 0x7efe60159240 2023-03-03T06:38:23.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe65551700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe60070300 con 0x7efe600cdc50 2023-03-03T06:38:23.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe65551700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6015dd40 con 0x7efe600ce150 2023-03-03T06:38:23.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5ffff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efe60159240 0x7efe600cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36810/0 (socket says 172.21.15.3:36810) 2023-03-03T06:38:23.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5ffff700 1 -- 172.21.15.3:0/2996252671 learned_addr learned my addr 172.21.15.3:0/2996252671 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:38:23.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe64d50700 1 --2- 172.21.15.3:0/2996252671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe600ce150 0x7efe600ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2224853926 0 0) 0x7efe60070020 con 0x7efe60159240 2023-03-03T06:38:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe4c0035d0 con 0x7efe60159240 2023-03-03T06:38:23.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3502972358 0 0) 0x7efe60070300 con 0x7efe600cdc50 2023-03-03T06:38:23.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe60070020 con 0x7efe600cdc50 2023-03-03T06:38:23.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1978497896 0 0) 0x7efe4c0035d0 con 0x7efe60159240 2023-03-03T06:38:23.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe600ce150 msgr2=0x7efe600ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 --2- 172.21.15.3:0/2996252671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe600ce150 0x7efe600ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 >> v1:172.21.15.32:6789/0 conn(0x7efe600cdc50 legacy=0x7efe6015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6015ce30 con 0x7efe60159240 2023-03-03T06:38:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7efe50005140 con 0x7efe60159240 2023-03-03T06:38:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 >> v1:172.21.15.3:6789/0 conn(0x7efe60159240 legacy=0x7efe600cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 --2- 172.21.15.3:0/2996252671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe600ce150 0x7efe4c003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 --2- 172.21.15.3:0/2996252671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe4c004190 0x7efe4c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 --2- 172.21.15.3:0/2996252671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe4c004db0 0x7efe4c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6015dd40 con 0x7efe4c004db0 2023-03-03T06:38:23.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c00ddd0 con 0x7efe4c004190 2023-03-03T06:38:23.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c00df90 con 0x7efe600ce150 2023-03-03T06:38:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.029+0000 7efe64d50700 1 --2- 172.21.15.3:0/2996252671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe4c004db0 0x7efe4c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe5ffff700 1 --2- 172.21.15.3:0/2996252671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe600ce150 0x7efe4c003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe57fff700 1 --2- 172.21.15.3:0/2996252671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe4c004190 0x7efe4c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe64d50700 1 -- 172.21.15.3:0/2996252671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe600ce150 msgr2=0x7efe4c003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe64d50700 1 --2- 172.21.15.3:0/2996252671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe600ce150 0x7efe4c003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe64d50700 1 -- 172.21.15.3:0/2996252671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe4c004190 msgr2=0x7efe4c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe64d50700 1 --2- 172.21.15.3:0/2996252671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe4c004190 0x7efe4c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe64d50700 1 -- 172.21.15.3:0/2996252671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0035d0 con 0x7efe4c004db0 2023-03-03T06:38:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe5ffff700 1 --2- 172.21.15.3:0/2996252671 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe600ce150 0x7efe4c003be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:38:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe64d50700 1 --2- 172.21.15.3:0/2996252671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe4c004db0 0x7efe4c005220 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efe58009fc0 tx=0x7efe4c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6503f23638175578 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5800ec30 con 0x7efe4c004db0 2023-03-03T06:38:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5800f040 con 0x7efe4c004db0 2023-03-03T06:38:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe5f7fe700 1 -- 172.21.15.3:0/2996252671 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58014480 con 0x7efe4c004db0 2023-03-03T06:38:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe65551700 1 -- 172.21.15.3:0/2996252671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe4c004db0 msgr2=0x7efe4c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe65551700 1 --2- 172.21.15.3:0/2996252671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe4c004db0 0x7efe4c005220 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efe58009fc0 tx=0x7efe4c0058a0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe64d50700 1 -- 172.21.15.3:0/2996252671 reap_dead start 2023-03-03T06:38:23.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe65551700 1 -- 172.21.15.3:0/2996252671 shutdown_connections 2023-03-03T06:38:23.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.033+0000 7efe65551700 1 -- 172.21.15.3:0/2996252671 >> 172.21.15.3:0/2996252671 conn(0x7efe60069d70 msgr2=0x7efe6006a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:23.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.037+0000 7efe65551700 1 -- 172.21.15.3:0/2996252671 shutdown_connections 2023-03-03T06:38:23.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.037+0000 7efe65551700 1 -- 172.21.15.3:0/2996252671 wait complete. 2023-03-03T06:38:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe65551700 1 Processor -- start 2023-03-03T06:38:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe65551700 1 -- start start 2023-03-03T06:38:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe65551700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe4c004db0 0x7efe6015b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe65551700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6014a320 0x7efe600b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe65551700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6014d380 0x7efe6014d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe65551700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6014dd40 con 0x7efe4c004db0 2023-03-03T06:38:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe65551700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6015bb60 con 0x7efe6014d380 2023-03-03T06:38:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe65551700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6015be40 con 0x7efe6014a320 2023-03-03T06:38:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe5ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe4c004db0 0x7efe6015b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe5ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe4c004db0 0x7efe6015b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51012/0 (socket says 172.21.15.3:51012) 2023-03-03T06:38:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe5ffff700 1 -- 172.21.15.3:0/552039769 learned_addr learned my addr 172.21.15.3:0/552039769 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe57fff700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6014a320 0x7efe600b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe64d50700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6014d380 0x7efe6014d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe5ffff700 1 -- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6014a320 msgr2=0x7efe600b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe5ffff700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6014a320 0x7efe600b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe5ffff700 1 -- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6014d380 msgr2=0x7efe6014d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe5ffff700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6014d380 0x7efe6014d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.041+0000 7efe5ffff700 1 -- 172.21.15.3:0/552039769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe60159610 con 0x7efe4c004db0 2023-03-03T06:38:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.045+0000 7efe5ffff700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe4c004db0 0x7efe6015b4a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efe5000e760 tx=0x7efe5000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.045+0000 7efe5d7fa700 1 -- 172.21.15.3:0/552039769 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe500072e0 con 0x7efe4c004db0 2023-03-03T06:38:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.045+0000 7efe5d7fa700 1 -- 172.21.15.3:0/552039769 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe50007440 con 0x7efe4c004db0 2023-03-03T06:38:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.045+0000 7efe65551700 1 -- 172.21.15.3:0/552039769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6015d110 con 0x7efe4c004db0 2023-03-03T06:38:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.045+0000 7efe5d7fa700 1 -- 172.21.15.3:0/552039769 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe50005960 con 0x7efe4c004db0 2023-03-03T06:38:23.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.045+0000 7efe65551700 1 -- 172.21.15.3:0/552039769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe600705e0 con 0x7efe4c004db0 2023-03-03T06:38:23.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.045+0000 7efe65551700 1 -- 172.21.15.3:0/552039769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe600c41e0 con 0x7efe4c004db0 2023-03-03T06:38:23.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.049+0000 7efe5d7fa700 1 -- 172.21.15.3:0/552039769 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7efe50005ac0 con 0x7efe4c004db0 2023-03-03T06:38:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.049+0000 7efe5d7fa700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efe40065000 0x7efe400674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.049+0000 7efe5d7fa700 1 -- 172.21.15.3:0/552039769 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7efe50082400 con 0x7efe4c004db0 2023-03-03T06:38:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.053+0000 7efe57fff700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efe40065000 0x7efe400674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.053+0000 7efe57fff700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efe40065000 0x7efe400674b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efe48007fd0 tx=0x7efe48009040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.057+0000 7efe5d7fa700 1 -- 172.21.15.3:0/552039769 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7efe600c41e0 con 0x7efe4c004db0 2023-03-03T06:38:23.266 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.253+0000 7efe65551700 1 -- 172.21.15.3:0/552039769 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efe6015bfb0 con 0x7efe40065000 2023-03-03T06:38:23.418 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.409+0000 7efe5d7fa700 1 -- 172.21.15.3:0/552039769 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7efe6015bfb0 con 0x7efe40065000 2023-03-03T06:38:23.419 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-03-03T06:38:23.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.413+0000 7efe65551700 1 -- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efe40065000 msgr2=0x7efe400674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:23.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.413+0000 7efe65551700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efe40065000 0x7efe400674b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efe48007fd0 tx=0x7efe48009040 comp rx=0 tx=0).stop 2023-03-03T06:38:23.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.413+0000 7efe65551700 1 -- 172.21.15.3:0/552039769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe4c004db0 msgr2=0x7efe6015b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:23.423 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.413+0000 7efe65551700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe4c004db0 0x7efe6015b4a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efe5000e760 tx=0x7efe5000eb20 comp rx=0 tx=0).stop 2023-03-03T06:38:23.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.413+0000 7efe65551700 1 -- 172.21.15.3:0/552039769 shutdown_connections 2023-03-03T06:38:23.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.413+0000 7efe65551700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efe40065000 0x7efe400674b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.413+0000 7efe65551700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe6014d380 0x7efe6014d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.424 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.413+0000 7efe65551700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efe6014a320 0x7efe600b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.413+0000 7efe65551700 1 --2- 172.21.15.3:0/552039769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe4c004db0 0x7efe6015b4a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.413+0000 7efe65551700 1 -- 172.21.15.3:0/552039769 >> 172.21.15.3:0/552039769 conn(0x7efe60069d70 msgr2=0x7efe600cc210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:23.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.413+0000 7efe65551700 1 -- 172.21.15.3:0/552039769 shutdown_connections 2023-03-03T06:38:23.425 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.413+0000 7efe65551700 1 -- 172.21.15.3:0/552039769 wait complete. 2023-03-03T06:38:23.463 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-03-03T06:38:23.463 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir3 1M 2023-03-03T06:38:23.690 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.681+0000 7f37fa00f700 1 Processor -- start 2023-03-03T06:38:23.691 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.681+0000 7f37fa00f700 1 -- start start 2023-03-03T06:38:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37fa00f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f4158e80 0x7f37f40cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37fa00f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37f4159250 0x7f37f406d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37fa00f700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f406ddc0 con 0x7f37f415d680 2023-03-03T06:38:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37fa00f700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f40cdf30 con 0x7f37f4159250 2023-03-03T06:38:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37fa00f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40ce210 con 0x7f37f4158e80 2023-03-03T06:38:23.696 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37f900d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f4158e80 0x7f37f40cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37f980e700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f37f415d680 0x7f37f406c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:60644/0 (socket says 172.21.15.3:60644) 2023-03-03T06:38:23.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37f980e700 1 -- 172.21.15.3:0/3031507342 learned_addr learned my addr 172.21.15.3:0/3031507342 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:38:23.697 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37f880c700 1 --2- 172.21.15.3:0/3031507342 >> v2:172.21.15.32:3300/0 conn(0x7f37f4159250 0x7f37f406d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37ebfff700 1 -- 172.21.15.3:0/3031507342 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101739026 0 0) 0x7f37f406ddc0 con 0x7f37f415d680 2023-03-03T06:38:23.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37ebfff700 1 -- 172.21.15.3:0/3031507342 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e00035d0 con 0x7f37f415d680 2023-03-03T06:38:23.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37f900d700 1 -- 172.21.15.3:0/3031507342 >> v2:172.21.15.32:3300/0 conn(0x7f37f4159250 msgr2=0x7f37f406d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:23.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37f900d700 1 --2- 172.21.15.3:0/3031507342 >> v2:172.21.15.32:3300/0 conn(0x7f37f4159250 0x7f37f406d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37f900d700 1 -- 172.21.15.3:0/3031507342 >> v1:172.21.15.32:6789/0 conn(0x7f37f415d680 legacy=0x7f37f406c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:23.698 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.685+0000 7f37f900d700 1 -- 172.21.15.3:0/3031507342 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f415ce80 con 0x7f37f4158e80 2023-03-03T06:38:23.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.689+0000 7f37f900d700 1 --2- 172.21.15.3:0/3031507342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f4158e80 0x7f37f40cd900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009850 tx=0x7f37f0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=603f3774aba85025 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:23.699 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.689+0000 7f37ebfff700 1 -- 172.21.15.3:0/3031507342 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0011830 con 0x7f37f4158e80 2023-03-03T06:38:23.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.689+0000 7f37ebfff700 1 -- 172.21.15.3:0/3031507342 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f0011990 con 0x7f37f4158e80 2023-03-03T06:38:23.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.689+0000 7f37ebfff700 1 -- 172.21.15.3:0/3031507342 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0011d40 con 0x7f37f4158e80 2023-03-03T06:38:23.700 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.689+0000 7f37fa00f700 1 -- 172.21.15.3:0/3031507342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f4158e80 msgr2=0x7f37f40cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:23.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.689+0000 7f37fa00f700 1 --2- 172.21.15.3:0/3031507342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f4158e80 0x7f37f40cd900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009850 tx=0x7f37f0009b60 comp rx=0 tx=0).stop 2023-03-03T06:38:23.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.689+0000 7f37fa00f700 1 -- 172.21.15.3:0/3031507342 shutdown_connections 2023-03-03T06:38:23.701 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.689+0000 7f37fa00f700 1 --2- 172.21.15.3:0/3031507342 >> v2:172.21.15.32:3300/0 conn(0x7f37f4159250 0x7f37f406d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.689+0000 7f37fa00f700 1 --2- 172.21.15.3:0/3031507342 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f4158e80 0x7f37f40cd900 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.689+0000 7f37fa00f700 1 -- 172.21.15.3:0/3031507342 >> 172.21.15.3:0/3031507342 conn(0x7f37f40cfc90 msgr2=0x7f37f40cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:23.702 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.693+0000 7f37fa00f700 1 -- 172.21.15.3:0/3031507342 shutdown_connections 2023-03-03T06:38:23.703 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.693+0000 7f37fa00f700 1 -- 172.21.15.3:0/3031507342 wait complete. 2023-03-03T06:38:23.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.693+0000 7f37fa00f700 1 Processor -- start 2023-03-03T06:38:23.705 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.693+0000 7f37fa00f700 1 -- start start 2023-03-03T06:38:23.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37fa00f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37f4158e80 0x7f37f40af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37fa00f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f4159250 0x7f37f41f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37fa00f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f415d680 0x7f37f41f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37fa00f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f41f6b00 con 0x7f37f4159250 2023-03-03T06:38:23.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37fa00f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f41f8cb0 con 0x7f37f4158e80 2023-03-03T06:38:23.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37fa00f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40afda0 con 0x7f37f415d680 2023-03-03T06:38:23.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37f880c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f4159250 0x7f37f41f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.708 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37f880c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f4159250 0x7f37f41f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51018/0 (socket says 172.21.15.3:51018) 2023-03-03T06:38:23.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37f880c700 1 -- 172.21.15.3:0/1335096648 learned_addr learned my addr 172.21.15.3:0/1335096648 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:23.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37f900d700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37f4158e80 0x7f37f40af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37f980e700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f415d680 0x7f37f41f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37f880c700 1 -- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f415d680 msgr2=0x7f37f41f5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:23.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37f880c700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f415d680 0x7f37f41f5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37f880c700 1 -- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37f4158e80 msgr2=0x7f37f40af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:23.709 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37f880c700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37f4158e80 0x7f37f40af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:23.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37f880c700 1 -- 172.21.15.3:0/1335096648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f00094b0 con 0x7f37f4159250 2023-03-03T06:38:23.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37f880c700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f4159250 0x7f37f41f58e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f37e400e9d0 tx=0x7f37e400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:23.710 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37e9ffb700 1 -- 172.21.15.3:0/1335096648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4009720 con 0x7f37f4159250 2023-03-03T06:38:23.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37e9ffb700 1 -- 172.21.15.3:0/1335096648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e400c850 con 0x7f37f4159250 2023-03-03T06:38:23.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37e9ffb700 1 -- 172.21.15.3:0/1335096648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e400cbc0 con 0x7f37f4159250 2023-03-03T06:38:23.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.697+0000 7f37fa00f700 1 -- 172.21.15.3:0/1335096648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f4159660 con 0x7f37f4159250 2023-03-03T06:38:23.711 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.701+0000 7f37fa00f700 1 -- 172.21.15.3:0/1335096648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f415d160 con 0x7f37f4159250 2023-03-03T06:38:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.701+0000 7f37fa00f700 1 -- 172.21.15.3:0/1335096648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f40ce4f0 con 0x7f37f4159250 2023-03-03T06:38:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.705+0000 7f37e9ffb700 1 -- 172.21.15.3:0/1335096648 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f37e4019030 con 0x7f37f4159250 2023-03-03T06:38:23.716 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.705+0000 7f37e9ffb700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f37d8065000 0x7f37d80674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:23.717 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.705+0000 7f37e9ffb700 1 -- 172.21.15.3:0/1335096648 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f37e40823b0 con 0x7f37f4159250 2023-03-03T06:38:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.709+0000 7f37f900d700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f37d8065000 0x7f37d80674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:23.720 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.709+0000 7f37f900d700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f37d8065000 0x7f37d80674b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f37f00090b0 tx=0x7f37f001c040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.713+0000 7f37e9ffb700 1 -- 172.21.15.3:0/1335096648 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f37e4050080 con 0x7f37f4159250 2023-03-03T06:38:23.921 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:23.909+0000 7f37fa00f700 1 -- 172.21.15.3:0/1335096648 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f37f4073e90 con 0x7f37d8065000 2023-03-03T06:38:24.075 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.065+0000 7f37e9ffb700 1 -- 172.21.15.3:0/1335096648 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f37f4073e90 con 0x7f37d8065000 2023-03-03T06:38:24.076 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-03-03T06:38:24.081 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.069+0000 7f37fa00f700 1 -- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f37d8065000 msgr2=0x7f37d80674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:24.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.069+0000 7f37fa00f700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f37d8065000 0x7f37d80674b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f37f00090b0 tx=0x7f37f001c040 comp rx=0 tx=0).stop 2023-03-03T06:38:24.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.069+0000 7f37fa00f700 1 -- 172.21.15.3:0/1335096648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f4159250 msgr2=0x7f37f41f58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:24.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.069+0000 7f37fa00f700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f4159250 0x7f37f41f58e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f37e400e9d0 tx=0x7f37e400ece0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.069+0000 7f37fa00f700 1 -- 172.21.15.3:0/1335096648 shutdown_connections 2023-03-03T06:38:24.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.069+0000 7f37fa00f700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f37d8065000 0x7f37d80674b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.069+0000 7f37fa00f700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f37f415d680 0x7f37f41f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.069+0000 7f37fa00f700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f4159250 0x7f37f41f58e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.082 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.069+0000 7f37fa00f700 1 --2- 172.21.15.3:0/1335096648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37f4158e80 0x7f37f40af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.069+0000 7f37fa00f700 1 -- 172.21.15.3:0/1335096648 >> 172.21.15.3:0/1335096648 conn(0x7f37f40cfc90 msgr2=0x7f37f406e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:24.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.069+0000 7f37fa00f700 1 -- 172.21.15.3:0/1335096648 shutdown_connections 2023-03-03T06:38:24.083 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.069+0000 7f37fa00f700 1 -- 172.21.15.3:0/1335096648 wait complete. 2023-03-03T06:38:24.116 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-03-03T06:38:24.117 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir4 1M 2023-03-03T06:38:24.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.317+0000 7f004cfa8700 1 Processor -- start 2023-03-03T06:38:24.328 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.317+0000 7f004cfa8700 1 -- start start 2023-03-03T06:38:24.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f004cfa8700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0048156ba0 0x7f0048071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f004cfa8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00480cf3b0 con 0x7f00480d0b30 2023-03-03T06:38:24.331 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f004cfa8700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0048156f70 con 0x7f004815d6e0 2023-03-03T06:38:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f004cfa8700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00480cd320 con 0x7f0048156ba0 2023-03-03T06:38:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f0047fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f00480d0b30 0x7f00480b46b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36826/0 (socket says 172.21.15.3:36826) 2023-03-03T06:38:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f0047fff700 1 -- 172.21.15.3:0/3146624959 learned_addr learned my addr 172.21.15.3:0/3146624959 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:38:24.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f00477fe700 1 --2- 172.21.15.3:0/3146624959 >> v2:172.21.15.32:3300/0 conn(0x7f0048156ba0 0x7f0048071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587726948 0 0) 0x7f00480cf3b0 con 0x7f00480d0b30 2023-03-03T06:38:24.333 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00300035d0 con 0x7f00480d0b30 2023-03-03T06:38:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884700308 0 0) 0x7f0048156f70 con 0x7f004815d6e0 2023-03-03T06:38:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00480cf3b0 con 0x7f004815d6e0 2023-03-03T06:38:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2618432826 0 0) 0x7f00300035d0 con 0x7f00480d0b30 2023-03-03T06:38:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 >> v2:172.21.15.32:3300/0 conn(0x7f0048156ba0 msgr2=0x7f0048071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f00467fc700 1 --2- 172.21.15.3:0/3146624959 >> v2:172.21.15.32:3300/0 conn(0x7f0048156ba0 0x7f0048071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 >> v1:172.21.15.32:6789/0 conn(0x7f004815d6e0 legacy=0x7f00480cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00480cd600 con 0x7f00480d0b30 2023-03-03T06:38:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f00400019f0 con 0x7f00480d0b30 2023-03-03T06:38:24.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.321+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 >> v1:172.21.15.3:6789/0 conn(0x7f00480d0b30 legacy=0x7f00480b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:24.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f00467fc700 1 --2- 172.21.15.3:0/3146624959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0030003bb0 0x7f0030003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f00467fc700 1 --2- 172.21.15.3:0/3146624959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0030004870 0x7f0030004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f00467fc700 1 --2- 172.21.15.3:0/3146624959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0030005220 0x7f003000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00480cd320 con 0x7f0030004870 2023-03-03T06:38:24.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003000c080 con 0x7f0030005220 2023-03-03T06:38:24.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003000c240 con 0x7f0030003bb0 2023-03-03T06:38:24.336 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f0046ffd700 1 --2- 172.21.15.3:0/3146624959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0030004870 0x7f0030004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.337 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f00477fe700 1 --2- 172.21.15.3:0/3146624959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0030003bb0 0x7f0030003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f0047fff700 1 --2- 172.21.15.3:0/3146624959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0030005220 0x7f003000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f0046ffd700 1 -- 172.21.15.3:0/3146624959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0030003bb0 msgr2=0x7f0030003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:24.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f0046ffd700 1 --2- 172.21.15.3:0/3146624959 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0030003bb0 0x7f0030003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f0046ffd700 1 -- 172.21.15.3:0/3146624959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0030005220 msgr2=0x7f003000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:24.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f0046ffd700 1 --2- 172.21.15.3:0/3146624959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0030005220 0x7f003000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.338 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f0046ffd700 1 -- 172.21.15.3:0/3146624959 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00300035d0 con 0x7f0030004870 2023-03-03T06:38:24.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f0047fff700 1 -- 172.21.15.3:0/3146624959 reap_dead start 2023-03-03T06:38:24.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f0046ffd700 1 --2- 172.21.15.3:0/3146624959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0030004870 0x7f0030004ce0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f003c001960 tx=0x7f00300058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b62af014a28674 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:24.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c001450 con 0x7f0030004870 2023-03-03T06:38:24.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c006aa0 con 0x7f0030004870 2023-03-03T06:38:24.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f00467fc700 1 -- 172.21.15.3:0/3146624959 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c006e50 con 0x7f0030004870 2023-03-03T06:38:24.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f004cfa8700 1 -- 172.21.15.3:0/3146624959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0030004870 msgr2=0x7f0030004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:24.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.325+0000 7f004cfa8700 1 --2- 172.21.15.3:0/3146624959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0030004870 0x7f0030004ce0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f003c001960 tx=0x7f00300058a0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.329+0000 7f004cfa8700 1 -- 172.21.15.3:0/3146624959 shutdown_connections 2023-03-03T06:38:24.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.329+0000 7f004cfa8700 1 --2- 172.21.15.3:0/3146624959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0030004870 0x7f0030004ce0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.329+0000 7f004cfa8700 1 -- 172.21.15.3:0/3146624959 >> 172.21.15.3:0/3146624959 conn(0x7f004806e5c0 msgr2=0x7f004806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:24.340 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.329+0000 7f004cfa8700 1 -- 172.21.15.3:0/3146624959 shutdown_connections 2023-03-03T06:38:24.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.329+0000 7f004cfa8700 1 -- 172.21.15.3:0/3146624959 wait complete. 2023-03-03T06:38:24.341 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.329+0000 7f004cfa8700 1 Processor -- start 2023-03-03T06:38:24.342 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f004cfa8700 1 -- start start 2023-03-03T06:38:24.343 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f004cfa8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0030004870 0x7f0048155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f004cfa8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00480b4c20 0x7f00480b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f004cfa8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048158370 0x7f00481587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f004cfa8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00480b55d0 con 0x7f0048158370 2023-03-03T06:38:24.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f004cfa8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00481558c0 con 0x7f0030004870 2023-03-03T06:38:24.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f004cfa8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00481590e0 con 0x7f00480b4c20 2023-03-03T06:38:24.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f0047fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048158370 0x7f00481587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f0047fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048158370 0x7f00481587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51044/0 (socket says 172.21.15.3:51044) 2023-03-03T06:38:24.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f0047fff700 1 -- 172.21.15.3:0/106488057 learned_addr learned my addr 172.21.15.3:0/106488057 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:24.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f00477fe700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0030004870 0x7f0048155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f0046ffd700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00480b4c20 0x7f00480b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f0047fff700 1 -- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00480b4c20 msgr2=0x7f00480b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:24.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f0047fff700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00480b4c20 0x7f00480b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f0047fff700 1 -- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0030004870 msgr2=0x7f0048155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:24.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f0047fff700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0030004870 0x7f0048155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f0047fff700 1 -- 172.21.15.3:0/106488057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00480cef40 con 0x7f0048158370 2023-03-03T06:38:24.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f00477fe700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0030004870 0x7f0048155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:38:24.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f0046ffd700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00480b4c20 0x7f00480b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:38:24.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f0047fff700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048158370 0x7f00481587e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f00480b57c0 tx=0x7f0040007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:24.347 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f002ffff700 1 -- 172.21.15.3:0/106488057 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004000ea00 con 0x7f0048158370 2023-03-03T06:38:24.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f002ffff700 1 -- 172.21.15.3:0/106488057 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004000eb80 con 0x7f0048158370 2023-03-03T06:38:24.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f004cfa8700 1 -- 172.21.15.3:0/106488057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00480cd8e0 con 0x7f0048158370 2023-03-03T06:38:24.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.333+0000 7f002ffff700 1 -- 172.21.15.3:0/106488057 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040013420 con 0x7f0048158370 2023-03-03T06:38:24.348 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.337+0000 7f004cfa8700 1 -- 172.21.15.3:0/106488057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00480cf0d0 con 0x7f0048158370 2023-03-03T06:38:24.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.337+0000 7f004cfa8700 1 -- 172.21.15.3:0/106488057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0048073e90 con 0x7f0048158370 2023-03-03T06:38:24.349 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.337+0000 7f002ffff700 1 -- 172.21.15.3:0/106488057 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f0040013580 con 0x7f0048158370 2023-03-03T06:38:24.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.341+0000 7f002ffff700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0034065030 0x7f00340674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.351 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.341+0000 7f002ffff700 1 -- 172.21.15.3:0/106488057 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f0040082840 con 0x7f0048158370 2023-03-03T06:38:24.352 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.341+0000 7f00477fe700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0034065030 0x7f00340674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.354 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.345+0000 7f00477fe700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0034065030 0x7f00340674e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f004804f390 tx=0x7f0038008040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:24.359 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.349+0000 7f002ffff700 1 -- 172.21.15.3:0/106488057 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f0040050540 con 0x7f0048158370 2023-03-03T06:38:24.557 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.545+0000 7f004cfa8700 1 -- 172.21.15.3:0/106488057 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0048071060 con 0x7f0034065030 2023-03-03T06:38:24.707 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.697+0000 7f002ffff700 1 -- 172.21.15.3:0/106488057 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f0048071060 con 0x7f0034065030 2023-03-03T06:38:24.708 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-03-03T06:38:24.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.701+0000 7f004cfa8700 1 -- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0034065030 msgr2=0x7f00340674e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:24.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.701+0000 7f004cfa8700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0034065030 0x7f00340674e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f004804f390 tx=0x7f0038008040 comp rx=0 tx=0).stop 2023-03-03T06:38:24.712 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.701+0000 7f004cfa8700 1 -- 172.21.15.3:0/106488057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048158370 msgr2=0x7f00481587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:24.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.701+0000 7f004cfa8700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048158370 0x7f00481587e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f00480b57c0 tx=0x7f0040007c90 comp rx=0 tx=0).stop 2023-03-03T06:38:24.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.701+0000 7f004cfa8700 1 -- 172.21.15.3:0/106488057 shutdown_connections 2023-03-03T06:38:24.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.701+0000 7f004cfa8700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0034065030 0x7f00340674e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.713 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.701+0000 7f004cfa8700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0048158370 0x7f00481587e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.701+0000 7f004cfa8700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f00480b4c20 0x7f00480b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.701+0000 7f004cfa8700 1 --2- 172.21.15.3:0/106488057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0030004870 0x7f0048155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.701+0000 7f004cfa8700 1 -- 172.21.15.3:0/106488057 >> 172.21.15.3:0/106488057 conn(0x7f004806e5c0 msgr2=0x7f004815e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:24.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.705+0000 7f004cfa8700 1 -- 172.21.15.3:0/106488057 shutdown_connections 2023-03-03T06:38:24.714 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.705+0000 7f004cfa8700 1 -- 172.21.15.3:0/106488057 wait complete. 2023-03-03T06:38:24.751 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-03-03T06:38:24.752 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir5 1M 2023-03-03T06:38:24.966 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.957+0000 7fd959682700 1 Processor -- start 2023-03-03T06:38:24.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.957+0000 7fd959682700 1 -- start start 2023-03-03T06:38:24.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.957+0000 7fd959682700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd95415a170 0x7fd9540cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.957+0000 7fd959682700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd9540cdcd0 0x7fd9540ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.969 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.957+0000 7fd959682700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95415dd30 con 0x7fd95406fe10 2023-03-03T06:38:24.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.957+0000 7fd959682700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9540cc020 con 0x7fd95415a170 2023-03-03T06:38:24.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.957+0000 7fd959682700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9540ce650 con 0x7fd9540cdcd0 2023-03-03T06:38:24.970 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.957+0000 7fd953fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd9540cdcd0 0x7fd9540ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.961+0000 7fd953fff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd9540cdcd0 0x7fd9540ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:47344/0 (socket says 172.21.15.3:47344) 2023-03-03T06:38:24.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.961+0000 7fd953fff700 1 -- 172.21.15.3:0/3963527623 learned_addr learned my addr 172.21.15.3:0/3963527623 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:24.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.961+0000 7fd952ffd700 1 --2- 172.21.15.3:0/3963527623 >> v2:172.21.15.32:3300/0 conn(0x7fd95415a170 0x7fd9540cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.961+0000 7fd953fff700 1 -- 172.21.15.3:0/3963527623 >> v2:172.21.15.32:3300/0 conn(0x7fd95415a170 msgr2=0x7fd9540cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:24.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.961+0000 7fd953fff700 1 --2- 172.21.15.3:0/3963527623 >> v2:172.21.15.32:3300/0 conn(0x7fd95415a170 0x7fd9540cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.971 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.961+0000 7fd953fff700 1 -- 172.21.15.3:0/3963527623 >> v1:172.21.15.32:6789/0 conn(0x7fd95406fe10 legacy=0x7fd9540cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:24.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.961+0000 7fd953fff700 1 -- 172.21.15.3:0/3963527623 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd954070280 con 0x7fd9540cdcd0 2023-03-03T06:38:24.972 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.961+0000 7fd953fff700 1 --2- 172.21.15.3:0/3963527623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd9540cdcd0 0x7fd9540ce110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009e50 tx=0x7fd94c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87e8253714520a9 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:24.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.961+0000 7fd951ffb700 1 -- 172.21.15.3:0/3963527623 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00e070 con 0x7fd9540cdcd0 2023-03-03T06:38:24.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.961+0000 7fd951ffb700 1 -- 172.21.15.3:0/3963527623 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94c00b9c0 con 0x7fd9540cdcd0 2023-03-03T06:38:24.973 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.961+0000 7fd951ffb700 1 -- 172.21.15.3:0/3963527623 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00bd70 con 0x7fd9540cdcd0 2023-03-03T06:38:24.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.965+0000 7fd959682700 1 -- 172.21.15.3:0/3963527623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd9540cdcd0 msgr2=0x7fd9540ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:24.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.965+0000 7fd959682700 1 --2- 172.21.15.3:0/3963527623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd9540cdcd0 0x7fd9540ce110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009e50 tx=0x7fd94c00b330 comp rx=0 tx=0).stop 2023-03-03T06:38:24.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.965+0000 7fd959682700 1 -- 172.21.15.3:0/3963527623 shutdown_connections 2023-03-03T06:38:24.974 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.965+0000 7fd959682700 1 --2- 172.21.15.3:0/3963527623 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd9540cdcd0 0x7fd9540ce110 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.965+0000 7fd959682700 1 --2- 172.21.15.3:0/3963527623 >> v2:172.21.15.32:3300/0 conn(0x7fd95415a170 0x7fd9540cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.965+0000 7fd959682700 1 -- 172.21.15.3:0/3963527623 >> 172.21.15.3:0/3963527623 conn(0x7fd95406ad30 msgr2=0x7fd95406b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:24.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.965+0000 7fd959682700 1 -- 172.21.15.3:0/3963527623 shutdown_connections 2023-03-03T06:38:24.975 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.965+0000 7fd959682700 1 -- 172.21.15.3:0/3963527623 wait complete. 2023-03-03T06:38:24.978 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd959682700 1 Processor -- start 2023-03-03T06:38:24.979 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd959682700 1 -- start start 2023-03-03T06:38:24.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd959682700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95406fe10 0x7fd9541586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd959682700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd9540cdcd0 0x7fd9540c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd959682700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95415a170 0x7fd9540b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd959682700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95415b9c0 con 0x7fd95406fe10 2023-03-03T06:38:24.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd959682700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95415a920 con 0x7fd95415a170 2023-03-03T06:38:24.981 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd959682700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95415ac00 con 0x7fd9540cdcd0 2023-03-03T06:38:24.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd952ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95406fe10 0x7fd9541586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd952ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95406fe10 0x7fd9541586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51060/0 (socket says 172.21.15.3:51060) 2023-03-03T06:38:24.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd952ffd700 1 -- 172.21.15.3:0/728025354 learned_addr learned my addr 172.21.15.3:0/728025354 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:24.982 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd9527fc700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd9540cdcd0 0x7fd9540c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd953fff700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95415a170 0x7fd9540b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd952ffd700 1 -- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd9540cdcd0 msgr2=0x7fd9540c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:24.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd952ffd700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd9540cdcd0 0x7fd9540c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd952ffd700 1 -- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95415a170 msgr2=0x7fd9540b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:24.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd952ffd700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95415a170 0x7fd9540b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:24.983 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.969+0000 7fd952ffd700 1 -- 172.21.15.3:0/728025354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c009b30 con 0x7fd95406fe10 2023-03-03T06:38:24.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.973+0000 7fd952ffd700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95406fe10 0x7fd9541586d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd94400e760 tx=0x7fd94400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:24.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.973+0000 7fd93b7fe700 1 -- 172.21.15.3:0/728025354 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944013610 con 0x7fd95406fe10 2023-03-03T06:38:24.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.973+0000 7fd93b7fe700 1 -- 172.21.15.3:0/728025354 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd944013770 con 0x7fd95406fe10 2023-03-03T06:38:24.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.973+0000 7fd959682700 1 -- 172.21.15.3:0/728025354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd954070840 con 0x7fd95406fe10 2023-03-03T06:38:24.984 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.973+0000 7fd93b7fe700 1 -- 172.21.15.3:0/728025354 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944013ae0 con 0x7fd95406fe10 2023-03-03T06:38:24.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.973+0000 7fd959682700 1 -- 172.21.15.3:0/728025354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd954070560 con 0x7fd95406fe10 2023-03-03T06:38:24.985 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.973+0000 7fd959682700 1 -- 172.21.15.3:0/728025354 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd954070b20 con 0x7fd95406fe10 2023-03-03T06:38:24.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.977+0000 7fd93b7fe700 1 -- 172.21.15.3:0/728025354 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7fd944023070 con 0x7fd95406fe10 2023-03-03T06:38:24.989 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.977+0000 7fd93b7fe700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd940065000 0x7fd9400674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:24.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.981+0000 7fd93b7fe700 1 -- 172.21.15.3:0/728025354 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fd944082110 con 0x7fd95406fe10 2023-03-03T06:38:24.990 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.981+0000 7fd9527fc700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd940065000 0x7fd9400674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:24.992 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.981+0000 7fd9527fc700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd940065000 0x7fd9400674b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd9540b5dd0 tx=0x7fd948009040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:24.995 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:24.985+0000 7fd93b7fe700 1 -- 172.21.15.3:0/728025354 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd94404fde0 con 0x7fd95406fe10 2023-03-03T06:38:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.189+0000 7fd959682700 1 -- 172.21.15.3:0/728025354 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd95404e180 con 0x7fd940065000 2023-03-03T06:38:25.339 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.329+0000 7fd93b7fe700 1 -- 172.21.15.3:0/728025354 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd95404e180 con 0x7fd940065000 2023-03-03T06:38:25.340 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-03-03T06:38:25.344 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.333+0000 7fd959682700 1 -- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd940065000 msgr2=0x7fd9400674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.333+0000 7fd959682700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd940065000 0x7fd9400674b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd9540b5dd0 tx=0x7fd948009040 comp rx=0 tx=0).stop 2023-03-03T06:38:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.333+0000 7fd959682700 1 -- 172.21.15.3:0/728025354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95406fe10 msgr2=0x7fd9541586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.333+0000 7fd959682700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95406fe10 0x7fd9541586d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd94400e760 tx=0x7fd94400eb20 comp rx=0 tx=0).stop 2023-03-03T06:38:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.333+0000 7fd959682700 1 -- 172.21.15.3:0/728025354 shutdown_connections 2023-03-03T06:38:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.333+0000 7fd959682700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd940065000 0x7fd9400674b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.333+0000 7fd959682700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd95415a170 0x7fd9540b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.345 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.333+0000 7fd959682700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd9540cdcd0 0x7fd9540c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.333+0000 7fd959682700 1 --2- 172.21.15.3:0/728025354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd95406fe10 0x7fd9541586d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.333+0000 7fd959682700 1 -- 172.21.15.3:0/728025354 >> 172.21.15.3:0/728025354 conn(0x7fd95406ad30 msgr2=0x7fd9540b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:25.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.333+0000 7fd959682700 1 -- 172.21.15.3:0/728025354 shutdown_connections 2023-03-03T06:38:25.346 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.333+0000 7fd959682700 1 -- 172.21.15.3:0/728025354 wait complete. 2023-03-03T06:38:25.379 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-03-03T06:38:25.379 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir6 1M 2023-03-03T06:38:25.587 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.577+0000 7f3f1a526700 1 Processor -- start 2023-03-03T06:38:25.588 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.577+0000 7f3f1a526700 1 -- start start 2023-03-03T06:38:25.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f1a526700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f140cb520 0x7f3f1406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:25.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f1a526700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f140ccbc0 con 0x7f3f140b0ed0 2023-03-03T06:38:25.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f1a526700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f140cbae0 con 0x7f3f1415c370 2023-03-03T06:38:25.591 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f1a526700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f140ce520 con 0x7f3f140cb520 2023-03-03T06:38:25.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f18d23700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3f140b0ed0 0x7f3f141601d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36842/0 (socket says 172.21.15.3:36842) 2023-03-03T06:38:25.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f18d23700 1 -- 172.21.15.3:0/121544032 learned_addr learned my addr 172.21.15.3:0/121544032 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:38:25.592 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f19d25700 1 --2- 172.21.15.3:0/121544032 >> v2:172.21.15.32:3300/0 conn(0x7f3f140cb520 0x7f3f1406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:25.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480221709 0 0) 0x7f3f140ccbc0 con 0x7f3f140b0ed0 2023-03-03T06:38:25.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f000035d0 con 0x7f3f140b0ed0 2023-03-03T06:38:25.593 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845284840 0 0) 0x7f3f140cbae0 con 0x7f3f1415c370 2023-03-03T06:38:25.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f140ccbc0 con 0x7f3f1415c370 2023-03-03T06:38:25.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 765627557 0 0) 0x7f3f000035d0 con 0x7f3f140b0ed0 2023-03-03T06:38:25.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 >> v2:172.21.15.32:3300/0 conn(0x7f3f140cb520 msgr2=0x7f3f1406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:25.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 --2- 172.21.15.3:0/121544032 >> v2:172.21.15.32:3300/0 conn(0x7f3f140cb520 0x7f3f1406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 >> v1:172.21.15.32:6789/0 conn(0x7f3f1415c370 legacy=0x7f3f140b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:25.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f140cc500 con 0x7f3f140b0ed0 2023-03-03T06:38:25.594 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f3f04001670 con 0x7f3f140b0ed0 2023-03-03T06:38:25.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 >> v1:172.21.15.3:6789/0 conn(0x7f3f140b0ed0 legacy=0x7f3f141601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:25.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 --2- 172.21.15.3:0/121544032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f00003bb0 0x7f3f00003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:25.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 --2- 172.21.15.3:0/121544032 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f00004870 0x7f3f00004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:25.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 --2- 172.21.15.3:0/121544032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f00005220 0x7f3f0000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:25.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0000c050 con 0x7f3f00005220 2023-03-03T06:38:25.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0000c1c0 con 0x7f3f00003bb0 2023-03-03T06:38:25.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0000c380 con 0x7f3f00004870 2023-03-03T06:38:25.595 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f19d25700 1 --2- 172.21.15.3:0/121544032 >> v2:172.21.15.32:3300/0 conn(0x7f3f140cb520 0x7f3f1406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:38:25.596 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.581+0000 7f3f19524700 1 --2- 172.21.15.3:0/121544032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f00003bb0 0x7f3f00003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f19d25700 1 --2- 172.21.15.3:0/121544032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f00005220 0x7f3f0000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f18d23700 1 --2- 172.21.15.3:0/121544032 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f00004870 0x7f3f00004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f19524700 1 -- 172.21.15.3:0/121544032 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f00004870 msgr2=0x7f3f00004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f19524700 1 --2- 172.21.15.3:0/121544032 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f00004870 0x7f3f00004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f19524700 1 -- 172.21.15.3:0/121544032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f00005220 msgr2=0x7f3f0000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f19524700 1 --2- 172.21.15.3:0/121544032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f00005220 0x7f3f0000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.597 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f19524700 1 -- 172.21.15.3:0/121544032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f000035d0 con 0x7f3f00003bb0 2023-03-03T06:38:25.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f19d25700 1 --2- 172.21.15.3:0/121544032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f00005220 0x7f3f0000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:38:25.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f19d25700 1 -- 172.21.15.3:0/121544032 reap_dead start 2023-03-03T06:38:25.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f19524700 1 --2- 172.21.15.3:0/121544032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f00003bb0 0x7f3f00003fa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3f10001950 tx=0x7f3f000058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e5d8d16263fb9fe server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:25.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10001d80 con 0x7f3f00003bb0 2023-03-03T06:38:25.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f100052f0 con 0x7f3f00003bb0 2023-03-03T06:38:25.598 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f0bfff700 1 -- 172.21.15.3:0/121544032 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10001440 con 0x7f3f00003bb0 2023-03-03T06:38:25.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f1a526700 1 -- 172.21.15.3:0/121544032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f00003bb0 msgr2=0x7f3f00003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:25.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f1a526700 1 --2- 172.21.15.3:0/121544032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f00003bb0 0x7f3f00003fa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3f10001950 tx=0x7f3f000058a0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f1a526700 1 -- 172.21.15.3:0/121544032 shutdown_connections 2023-03-03T06:38:25.599 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f1a526700 1 --2- 172.21.15.3:0/121544032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f00003bb0 0x7f3f00003fa0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.585+0000 7f3f1a526700 1 -- 172.21.15.3:0/121544032 >> 172.21.15.3:0/121544032 conn(0x7f3f140607c0 msgr2=0x7f3f14060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:25.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.589+0000 7f3f1a526700 1 -- 172.21.15.3:0/121544032 shutdown_connections 2023-03-03T06:38:25.600 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.589+0000 7f3f1a526700 1 -- 172.21.15.3:0/121544032 wait complete. 2023-03-03T06:38:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.593+0000 7f3f1a526700 1 Processor -- start 2023-03-03T06:38:25.603 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.593+0000 7f3f1a526700 1 -- start start 2023-03-03T06:38:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.593+0000 7f3f1a526700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f00003bb0 0x7f3f14150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:25.604 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.593+0000 7f3f1a526700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f140b18b0 0x7f3f14159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.593+0000 7f3f1a526700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f140b1d20 0x7f3f14159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.593+0000 7f3f1a526700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1415d830 con 0x7f3f140b1d20 2023-03-03T06:38:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.593+0000 7f3f1a526700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f141f1d60 con 0x7f3f00003bb0 2023-03-03T06:38:25.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.593+0000 7f3f1a526700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1415ccb0 con 0x7f3f140b18b0 2023-03-03T06:38:25.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.593+0000 7f3f19d25700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f140b1d20 0x7f3f14159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:25.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.593+0000 7f3f19d25700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f140b1d20 0x7f3f14159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51084/0 (socket says 172.21.15.3:51084) 2023-03-03T06:38:25.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.593+0000 7f3f19d25700 1 -- 172.21.15.3:0/1416900110 learned_addr learned my addr 172.21.15.3:0/1416900110 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:25.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.593+0000 7f3f18d23700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f140b18b0 0x7f3f14159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:25.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.593+0000 7f3f19524700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f00003bb0 0x7f3f14150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:25.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.597+0000 7f3f19d25700 1 -- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f140b18b0 msgr2=0x7f3f14159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:25.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.597+0000 7f3f19d25700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f140b18b0 0x7f3f14159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.597+0000 7f3f19d25700 1 -- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f00003bb0 msgr2=0x7f3f14150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:25.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.597+0000 7f3f19d25700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f00003bb0 0x7f3f14150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.597+0000 7f3f19d25700 1 -- 172.21.15.3:0/1416900110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1415c740 con 0x7f3f140b1d20 2023-03-03T06:38:25.607 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.597+0000 7f3f19d25700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f140b1d20 0x7f3f14159cc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3f140cae20 tx=0x7f3f0c00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:25.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.597+0000 7f3f09ffb700 1 -- 172.21.15.3:0/1416900110 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00e040 con 0x7f3f140b1d20 2023-03-03T06:38:25.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.597+0000 7f3f09ffb700 1 -- 172.21.15.3:0/1416900110 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c00f040 con 0x7f3f140b1d20 2023-03-03T06:38:25.608 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.597+0000 7f3f1a526700 1 -- 172.21.15.3:0/1416900110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f140cc7e0 con 0x7f3f140b1d20 2023-03-03T06:38:25.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.597+0000 7f3f09ffb700 1 -- 172.21.15.3:0/1416900110 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c011ee0 con 0x7f3f140b1d20 2023-03-03T06:38:25.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.597+0000 7f3f1a526700 1 -- 172.21.15.3:0/1416900110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1406c090 con 0x7f3f140b1d20 2023-03-03T06:38:25.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.601+0000 7f3f1a526700 1 -- 172.21.15.3:0/1416900110 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f14073e90 con 0x7f3f140b1d20 2023-03-03T06:38:25.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.601+0000 7f3f09ffb700 1 -- 172.21.15.3:0/1416900110 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f3f0c003c60 con 0x7f3f140b1d20 2023-03-03T06:38:25.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.605+0000 7f3f09ffb700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3ef8065000 0x7f3ef80674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:25.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.605+0000 7f3f09ffb700 1 -- 172.21.15.3:0/1416900110 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f3f0c082b90 con 0x7f3f140b1d20 2023-03-03T06:38:25.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.605+0000 7f3f19524700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3ef8065000 0x7f3ef80674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:25.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.605+0000 7f3f19524700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3ef8065000 0x7f3ef80674b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f10005040 tx=0x7f3f10009030 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:25.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.613+0000 7f3f09ffb700 1 -- 172.21.15.3:0/1416900110 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f3f0c0508f0 con 0x7f3f140b1d20 2023-03-03T06:38:25.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.809+0000 7f3f1a526700 1 -- 172.21.15.3:0/1416900110 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f1415da70 con 0x7f3ef8065000 2023-03-03T06:38:25.980 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.969+0000 7f3f09ffb700 1 -- 172.21.15.3:0/1416900110 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f3f10005150 con 0x7f3ef8065000 2023-03-03T06:38:25.981 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-03-03T06:38:25.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.973+0000 7f3f1a526700 1 -- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3ef8065000 msgr2=0x7f3ef80674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:25.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.973+0000 7f3f1a526700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3ef8065000 0x7f3ef80674b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f10005040 tx=0x7f3f10009030 comp rx=0 tx=0).stop 2023-03-03T06:38:25.986 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.977+0000 7f3f1a526700 1 -- 172.21.15.3:0/1416900110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f140b1d20 msgr2=0x7f3f14159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:25.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.977+0000 7f3f1a526700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f140b1d20 0x7f3f14159cc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3f140cae20 tx=0x7f3f0c00f6c0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.977+0000 7f3f1a526700 1 -- 172.21.15.3:0/1416900110 shutdown_connections 2023-03-03T06:38:25.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.977+0000 7f3f1a526700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f3ef8065000 0x7f3ef80674b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.977+0000 7f3f1a526700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f140b1d20 0x7f3f14159cc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.987 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.977+0000 7f3f1a526700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f3f140b18b0 0x7f3f14159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.977+0000 7f3f1a526700 1 --2- 172.21.15.3:0/1416900110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f00003bb0 0x7f3f14150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:25.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.977+0000 7f3f1a526700 1 -- 172.21.15.3:0/1416900110 >> 172.21.15.3:0/1416900110 conn(0x7f3f140607c0 msgr2=0x7f3f1406ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:25.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.977+0000 7f3f1a526700 1 -- 172.21.15.3:0/1416900110 shutdown_connections 2023-03-03T06:38:25.988 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:25.977+0000 7f3f1a526700 1 -- 172.21.15.3:0/1416900110 wait complete. 2023-03-03T06:38:26.022 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-03-03T06:38:26.023 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir7 1M 2023-03-03T06:38:26.238 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.229+0000 7fc649220700 1 Processor -- start 2023-03-03T06:38:26.239 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.229+0000 7fc649220700 1 -- start start 2023-03-03T06:38:26.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.229+0000 7fc649220700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc64415d6e0 0x7fc6440cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.229+0000 7fc649220700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6440cfd80 0x7fc6440d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.229+0000 7fc649220700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6440cf3b0 con 0x7fc644156ba0 2023-03-03T06:38:26.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.229+0000 7fc649220700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc644156f70 con 0x7fc64415d6e0 2023-03-03T06:38:26.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.229+0000 7fc649220700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6440cd320 con 0x7fc6440cfd80 2023-03-03T06:38:26.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.229+0000 7fc6437fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc64415d6e0 0x7fc6440cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.229+0000 7fc6437fe700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc64415d6e0 0x7fc6440cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51096/0 (socket says 172.21.15.3:51096) 2023-03-03T06:38:26.242 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.229+0000 7fc6437fe700 1 -- 172.21.15.3:0/1183526346 learned_addr learned my addr 172.21.15.3:0/1183526346 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:26.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.233+0000 7fc648a1f700 1 --2- 172.21.15.3:0/1183526346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6440cfd80 0x7fc6440d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.233+0000 7fc6437fe700 1 -- 172.21.15.3:0/1183526346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6440cfd80 msgr2=0x7fc6440d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:26.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.233+0000 7fc6437fe700 1 --2- 172.21.15.3:0/1183526346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6440cfd80 0x7fc6440d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.243 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.233+0000 7fc6437fe700 1 -- 172.21.15.3:0/1183526346 >> v1:172.21.15.32:6789/0 conn(0x7fc644156ba0 legacy=0x7fc644071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:26.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.233+0000 7fc6437fe700 1 -- 172.21.15.3:0/1183526346 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6440cd600 con 0x7fc64415d6e0 2023-03-03T06:38:26.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.233+0000 7fc642ffd700 1 -- 172.21.15.3:0/1183526346 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036590658 0 0) 0x7fc6440cf3b0 con 0x7fc644156ba0 2023-03-03T06:38:26.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.233+0000 7fc6437fe700 1 --2- 172.21.15.3:0/1183526346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc64415d6e0 0x7fc6440cf840 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc638009910 tx=0x7fc638009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b269f1bac9555541 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:26.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.233+0000 7fc642ffd700 1 -- 172.21.15.3:0/1183526346 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63800ba60 con 0x7fc64415d6e0 2023-03-03T06:38:26.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.233+0000 7fc642ffd700 1 -- 172.21.15.3:0/1183526346 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63800bbc0 con 0x7fc64415d6e0 2023-03-03T06:38:26.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.233+0000 7fc642ffd700 1 -- 172.21.15.3:0/1183526346 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638009e60 con 0x7fc64415d6e0 2023-03-03T06:38:26.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.233+0000 7fc649220700 1 -- 172.21.15.3:0/1183526346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc64415d6e0 msgr2=0x7fc6440cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:26.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.233+0000 7fc649220700 1 --2- 172.21.15.3:0/1183526346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc64415d6e0 0x7fc6440cf840 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc638009910 tx=0x7fc638009c20 comp rx=0 tx=0).stop 2023-03-03T06:38:26.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.237+0000 7fc649220700 1 -- 172.21.15.3:0/1183526346 shutdown_connections 2023-03-03T06:38:26.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.237+0000 7fc649220700 1 --2- 172.21.15.3:0/1183526346 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6440cfd80 0x7fc6440d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.237+0000 7fc649220700 1 --2- 172.21.15.3:0/1183526346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc64415d6e0 0x7fc6440cf840 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.237+0000 7fc649220700 1 -- 172.21.15.3:0/1183526346 >> 172.21.15.3:0/1183526346 conn(0x7fc64406e5c0 msgr2=0x7fc64406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:26.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.237+0000 7fc649220700 1 -- 172.21.15.3:0/1183526346 shutdown_connections 2023-03-03T06:38:26.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.237+0000 7fc649220700 1 -- 172.21.15.3:0/1183526346 wait complete. 2023-03-03T06:38:26.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.237+0000 7fc649220700 1 Processor -- start 2023-03-03T06:38:26.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.237+0000 7fc649220700 1 -- start start 2023-03-03T06:38:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc649220700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6440cfd80 0x7fc64415f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc649220700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc644156ba0 0x7fc644150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc649220700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc64415d6e0 0x7fc6440b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc649220700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64415fb60 con 0x7fc644156ba0 2023-03-03T06:38:26.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc649220700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6440c6900 con 0x7fc64415d6e0 2023-03-03T06:38:26.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc649220700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6440c6be0 con 0x7fc6440cfd80 2023-03-03T06:38:26.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc6437fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc644156ba0 0x7fc644150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc6437fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc644156ba0 0x7fc644150d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51100/0 (socket says 172.21.15.3:51100) 2023-03-03T06:38:26.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc6437fe700 1 -- 172.21.15.3:0/3187788577 learned_addr learned my addr 172.21.15.3:0/3187788577 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:26.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc648a1f700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc64415d6e0 0x7fc6440b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.253 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc643fff700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6440cfd80 0x7fc64415f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc6437fe700 1 -- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6440cfd80 msgr2=0x7fc64415f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:26.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc6437fe700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6440cfd80 0x7fc64415f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc6437fe700 1 -- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc64415d6e0 msgr2=0x7fc6440b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:26.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc6437fe700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc64415d6e0 0x7fc6440b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc6437fe700 1 -- 172.21.15.3:0/3187788577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc638009570 con 0x7fc644156ba0 2023-03-03T06:38:26.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc643fff700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6440cfd80 0x7fc64415f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:38:26.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc6437fe700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc644156ba0 0x7fc644150d80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc638009c60 tx=0x7fc638004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:26.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc640ff9700 1 -- 172.21.15.3:0/3187788577 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638004220 con 0x7fc644156ba0 2023-03-03T06:38:26.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc640ff9700 1 -- 172.21.15.3:0/3187788577 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63801f070 con 0x7fc644156ba0 2023-03-03T06:38:26.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc640ff9700 1 -- 172.21.15.3:0/3187788577 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6380045a0 con 0x7fc644156ba0 2023-03-03T06:38:26.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc649220700 1 -- 172.21.15.3:0/3187788577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6440cef40 con 0x7fc644156ba0 2023-03-03T06:38:26.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.241+0000 7fc649220700 1 -- 172.21.15.3:0/3187788577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6440cd8e0 con 0x7fc644156ba0 2023-03-03T06:38:26.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.245+0000 7fc649220700 1 -- 172.21.15.3:0/3187788577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6440cf0d0 con 0x7fc644156ba0 2023-03-03T06:38:26.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.245+0000 7fc640ff9700 1 -- 172.21.15.3:0/3187788577 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7fc63800f050 con 0x7fc644156ba0 2023-03-03T06:38:26.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.249+0000 7fc640ff9700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc630065000 0x7fc6300674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.249+0000 7fc640ff9700 1 -- 172.21.15.3:0/3187788577 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fc638021070 con 0x7fc644156ba0 2023-03-03T06:38:26.260 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.249+0000 7fc643fff700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc630065000 0x7fc6300674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.249+0000 7fc643fff700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc630065000 0x7fc6300674b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc634000f60 tx=0x7fc634009040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.253+0000 7fc640ff9700 1 -- 172.21.15.3:0/3187788577 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc638051350 con 0x7fc644156ba0 2023-03-03T06:38:26.465 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.453+0000 7fc649220700 1 -- 172.21.15.3:0/3187788577 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc644073e90 con 0x7fc630065000 2023-03-03T06:38:26.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.613+0000 7fc640ff9700 1 -- 172.21.15.3:0/3187788577 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc644073e90 con 0x7fc630065000 2023-03-03T06:38:26.623 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-03-03T06:38:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.617+0000 7fc649220700 1 -- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc630065000 msgr2=0x7fc6300674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.617+0000 7fc649220700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc630065000 0x7fc6300674b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc634000f60 tx=0x7fc634009040 comp rx=0 tx=0).stop 2023-03-03T06:38:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.617+0000 7fc649220700 1 -- 172.21.15.3:0/3187788577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc644156ba0 msgr2=0x7fc644150d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.617+0000 7fc649220700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc644156ba0 0x7fc644150d80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc638009c60 tx=0x7fc638004770 comp rx=0 tx=0).stop 2023-03-03T06:38:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.617+0000 7fc649220700 1 -- 172.21.15.3:0/3187788577 shutdown_connections 2023-03-03T06:38:26.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.617+0000 7fc649220700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc630065000 0x7fc6300674b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.617+0000 7fc649220700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc64415d6e0 0x7fc6440b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.617+0000 7fc649220700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc644156ba0 0x7fc644150d80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.617+0000 7fc649220700 1 --2- 172.21.15.3:0/3187788577 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc6440cfd80 0x7fc64415f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.628 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.617+0000 7fc649220700 1 -- 172.21.15.3:0/3187788577 >> 172.21.15.3:0/3187788577 conn(0x7fc64406e5c0 msgr2=0x7fc6440d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:26.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.617+0000 7fc649220700 1 -- 172.21.15.3:0/3187788577 shutdown_connections 2023-03-03T06:38:26.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.617+0000 7fc649220700 1 -- 172.21.15.3:0/3187788577 wait complete. 2023-03-03T06:38:26.663 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-03-03T06:38:26.664 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir8 1M 2023-03-03T06:38:26.887 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.877+0000 7f753d4fd700 1 Processor -- start 2023-03-03T06:38:26.888 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.877+0000 7f753d4fd700 1 -- start start 2023-03-03T06:38:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f753d4fd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7538156ba0 0x7f7538071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.890 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f753d4fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f753815d6e0 0x7f75380cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f753ccfc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f75380cfd80 0x7f75380d0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36856/0 (socket says 172.21.15.3:36856) 2023-03-03T06:38:26.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f753ccfc700 1 -- 172.21.15.3:0/3487264808 learned_addr learned my addr 172.21.15.3:0/3487264808 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:38:26.891 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f753d4fd700 1 -- 172.21.15.3:0/3487264808 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75380cf3b0 con 0x7f75380cfd80 2023-03-03T06:38:26.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f7537fff700 1 --2- 172.21.15.3:0/3487264808 >> v2:172.21.15.32:3300/0 conn(0x7f7538156ba0 0x7f7538071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f753d4fd700 1 -- 172.21.15.3:0/3487264808 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7538156f70 con 0x7f7538156ba0 2023-03-03T06:38:26.892 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f753d4fd700 1 -- 172.21.15.3:0/3487264808 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75380cd320 con 0x7f753815d6e0 2023-03-03T06:38:26.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f752ffff700 1 --2- 172.21.15.3:0/3487264808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f753815d6e0 0x7f75380cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6736723 0 0) 0x7f75380cf3b0 con 0x7f75380cfd80 2023-03-03T06:38:26.893 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75240035d0 con 0x7f75380cfd80 2023-03-03T06:38:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1957738030 0 0) 0x7f75240035d0 con 0x7f75380cfd80 2023-03-03T06:38:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f753815d6e0 msgr2=0x7f75380cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 --2- 172.21.15.3:0/3487264808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f753815d6e0 0x7f75380cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 >> v2:172.21.15.32:3300/0 conn(0x7f7538156ba0 msgr2=0x7f7538071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 --2- 172.21.15.3:0/3487264808 >> v2:172.21.15.32:3300/0 conn(0x7f7538156ba0 0x7f7538071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.894 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75380cd600 con 0x7f75380cfd80 2023-03-03T06:38:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3262727075 0 0) 0x7f75300019f0 con 0x7f75380cfd80 2023-03-03T06:38:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 >> v1:172.21.15.3:6789/0 conn(0x7f75380cfd80 legacy=0x7f75380d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 --2- 172.21.15.3:0/3487264808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7524003a20 0x7f7524003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 --2- 172.21.15.3:0/3487264808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f753815d6e0 0x7f7524004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 --2- 172.21.15.3:0/3487264808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7524004db0 0x7f7524005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7538156f70 con 0x7f7524004db0 2023-03-03T06:38:26.895 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75380cd320 con 0x7f7524003a20 2023-03-03T06:38:26.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752400ba30 con 0x7f753815d6e0 2023-03-03T06:38:26.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f753ccfc700 1 --2- 172.21.15.3:0/3487264808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7524004db0 0x7f7524005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f7537fff700 1 --2- 172.21.15.3:0/3487264808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7524003a20 0x7f7524003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.896 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.881+0000 7f752ffff700 1 --2- 172.21.15.3:0/3487264808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f753815d6e0 0x7f7524004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f752ffff700 1 -- 172.21.15.3:0/3487264808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7524003a20 msgr2=0x7f7524003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:26.897 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f752ffff700 1 --2- 172.21.15.3:0/3487264808 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7524003a20 0x7f7524003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f752ffff700 1 -- 172.21.15.3:0/3487264808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7524004db0 msgr2=0x7f7524005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:26.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f752ffff700 1 --2- 172.21.15.3:0/3487264808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7524004db0 0x7f7524005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f752ffff700 1 -- 172.21.15.3:0/3487264808 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75240035d0 con 0x7f753815d6e0 2023-03-03T06:38:26.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f752ffff700 1 --2- 172.21.15.3:0/3487264808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f753815d6e0 0x7f7524004870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7520009b70 tx=0x7f75240058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1c1299d9a245323 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:26.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752000e6c0 con 0x7f753815d6e0 2023-03-03T06:38:26.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752000e820 con 0x7f753815d6e0 2023-03-03T06:38:26.898 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f75377fe700 1 -- 172.21.15.3:0/3487264808 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752000eb90 con 0x7f753815d6e0 2023-03-03T06:38:26.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f753d4fd700 1 -- 172.21.15.3:0/3487264808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f753815d6e0 msgr2=0x7f7524004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:26.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f753d4fd700 1 --2- 172.21.15.3:0/3487264808 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f753815d6e0 0x7f7524004870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7520009b70 tx=0x7f75240058a0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f753ccfc700 1 -- 172.21.15.3:0/3487264808 reap_dead start 2023-03-03T06:38:26.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f753d4fd700 1 -- 172.21.15.3:0/3487264808 shutdown_connections 2023-03-03T06:38:26.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f753d4fd700 1 -- 172.21.15.3:0/3487264808 >> 172.21.15.3:0/3487264808 conn(0x7f753806e5c0 msgr2=0x7f753806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:26.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f753d4fd700 1 -- 172.21.15.3:0/3487264808 shutdown_connections 2023-03-03T06:38:26.899 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.885+0000 7f753d4fd700 1 -- 172.21.15.3:0/3487264808 wait complete. 2023-03-03T06:38:26.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.889+0000 7f753d4fd700 1 Processor -- start 2023-03-03T06:38:26.900 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.889+0000 7f753d4fd700 1 -- start start 2023-03-03T06:38:26.901 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.889+0000 7f753d4fd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753815d6e0 0x7f7538151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.889+0000 7f753d4fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7538151050 0x7f75380c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.889+0000 7f753d4fd700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75380b8170 0x7f75380b85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.889+0000 7f753d4fd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75380b7d50 con 0x7f753815d6e0 2023-03-03T06:38:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.889+0000 7f753d4fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75380b6df0 con 0x7f7538151050 2023-03-03T06:38:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.889+0000 7f753d4fd700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75380b70d0 con 0x7f75380b8170 2023-03-03T06:38:26.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f7537fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753815d6e0 0x7f7538151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f7537fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753815d6e0 0x7f7538151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51126/0 (socket says 172.21.15.3:51126) 2023-03-03T06:38:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f7537fff700 1 -- 172.21.15.3:0/1061202365 learned_addr learned my addr 172.21.15.3:0/1061202365 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f752ffff700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7538151050 0x7f75380c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f753ccfc700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75380b8170 0x7f75380b85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f7537fff700 1 -- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75380b8170 msgr2=0x7f75380b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f7537fff700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75380b8170 0x7f75380b85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f7537fff700 1 -- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7538151050 msgr2=0x7f75380c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:26.904 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f7537fff700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7538151050 0x7f75380c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:26.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f7537fff700 1 -- 172.21.15.3:0/1061202365 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75380717c0 con 0x7f753815d6e0 2023-03-03T06:38:26.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f753ccfc700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75380b8170 0x7f75380b85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:38:26.905 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f7537fff700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753815d6e0 0x7f7538151550 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7528011a20 tx=0x7f7528011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:26.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f75357fa700 1 -- 172.21.15.3:0/1061202365 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752800fa00 con 0x7f753815d6e0 2023-03-03T06:38:26.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f75357fa700 1 -- 172.21.15.3:0/1061202365 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752800fb60 con 0x7f753815d6e0 2023-03-03T06:38:26.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f75357fa700 1 -- 172.21.15.3:0/1061202365 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7528014440 con 0x7f753815d6e0 2023-03-03T06:38:26.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f753d4fd700 1 -- 172.21.15.3:0/1061202365 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75380cd8e0 con 0x7f753815d6e0 2023-03-03T06:38:26.906 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.893+0000 7f753d4fd700 1 -- 172.21.15.3:0/1061202365 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75380cef40 con 0x7f753815d6e0 2023-03-03T06:38:26.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.897+0000 7f753d4fd700 1 -- 172.21.15.3:0/1061202365 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75380b7fc0 con 0x7f753815d6e0 2023-03-03T06:38:26.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.897+0000 7f75357fa700 1 -- 172.21.15.3:0/1061202365 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f75280145a0 con 0x7f753815d6e0 2023-03-03T06:38:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.901+0000 7f75357fa700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7518065000 0x7f75180674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:26.910 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.901+0000 7f75357fa700 1 -- 172.21.15.3:0/1061202365 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f7528082d40 con 0x7f753815d6e0 2023-03-03T06:38:26.911 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.901+0000 7f752ffff700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7518065000 0x7f75180674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:26.913 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.901+0000 7f752ffff700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7518065000 0x7f75180674b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75200090b0 tx=0x7f7520008030 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:26.915 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:26.905+0000 7f75357fa700 1 -- 172.21.15.3:0/1061202365 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7528050b00 con 0x7f753815d6e0 2023-03-03T06:38:27.113 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.101+0000 7f753d4fd700 1 -- 172.21.15.3:0/1061202365 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f75380b5350 con 0x7f7518065000 2023-03-03T06:38:27.267 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.257+0000 7f75357fa700 1 -- 172.21.15.3:0/1061202365 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f75380b5350 con 0x7f7518065000 2023-03-03T06:38:27.267 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-03-03T06:38:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.261+0000 7f753d4fd700 1 -- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7518065000 msgr2=0x7f75180674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.261+0000 7f753d4fd700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7518065000 0x7f75180674b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75200090b0 tx=0x7f7520008030 comp rx=0 tx=0).stop 2023-03-03T06:38:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.261+0000 7f753d4fd700 1 -- 172.21.15.3:0/1061202365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753815d6e0 msgr2=0x7f7538151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:27.272 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.261+0000 7f753d4fd700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753815d6e0 0x7f7538151550 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7528011a20 tx=0x7f7528011de0 comp rx=0 tx=0).stop 2023-03-03T06:38:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.261+0000 7f753d4fd700 1 -- 172.21.15.3:0/1061202365 shutdown_connections 2023-03-03T06:38:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.261+0000 7f753d4fd700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7518065000 0x7f75180674b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.261+0000 7f753d4fd700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f75380b8170 0x7f75380b85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.261+0000 7f753d4fd700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7538151050 0x7f75380c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.261+0000 7f753d4fd700 1 --2- 172.21.15.3:0/1061202365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753815d6e0 0x7f7538151550 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.261+0000 7f753d4fd700 1 -- 172.21.15.3:0/1061202365 >> 172.21.15.3:0/1061202365 conn(0x7f753806e5c0 msgr2=0x7f7538160940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:27.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.261+0000 7f753d4fd700 1 -- 172.21.15.3:0/1061202365 shutdown_connections 2023-03-03T06:38:27.274 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.261+0000 7f753d4fd700 1 -- 172.21.15.3:0/1061202365 wait complete. 2023-03-03T06:38:27.308 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-03-03T06:38:27.308 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir9 1M 2023-03-03T06:38:27.524 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.513+0000 7f768126a700 1 Processor -- start 2023-03-03T06:38:27.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.513+0000 7f768126a700 1 -- start start 2023-03-03T06:38:27.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.517+0000 7f768126a700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f767c0cad60 0x7f767c0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:27.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.517+0000 7f768126a700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767c0cab60 con 0x7f767c06f970 2023-03-03T06:38:27.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.517+0000 7f768126a700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767c06f4e0 con 0x7f767c158a00 2023-03-03T06:38:27.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.517+0000 7f768126a700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767c06ef00 con 0x7f767c0cad60 2023-03-03T06:38:27.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.517+0000 7f767bfff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f767c06f970 0x7f767c06b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36866/0 (socket says 172.21.15.3:36866) 2023-03-03T06:38:27.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.517+0000 7f767bfff700 1 -- 172.21.15.3:0/770276889 learned_addr learned my addr 172.21.15.3:0/770276889 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:38:27.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f767a7fc700 1 --2- 172.21.15.3:0/770276889 >> v2:172.21.15.3:3300/0 conn(0x7f767c0cad60 0x7f767c0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:27.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f7679ffb700 1 -- 172.21.15.3:0/770276889 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109717433 0 0) 0x7f767c06f4e0 con 0x7f767c158a00 2023-03-03T06:38:27.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f7679ffb700 1 -- 172.21.15.3:0/770276889 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76640034f0 con 0x7f767c158a00 2023-03-03T06:38:27.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f767a7fc700 1 -- 172.21.15.3:0/770276889 >> v1:172.21.15.32:6789/0 conn(0x7f767c158a00 legacy=0x7f767c0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:27.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f767a7fc700 1 -- 172.21.15.3:0/770276889 >> v1:172.21.15.3:6789/0 conn(0x7f767c06f970 legacy=0x7f767c06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:27.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f767a7fc700 1 -- 172.21.15.3:0/770276889 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c070870 con 0x7f767c0cad60 2023-03-03T06:38:27.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f767a7fc700 1 --2- 172.21.15.3:0/770276889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c0cad60 0x7f767c0b2540 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7670009850 tx=0x7f7670009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1df09a82c6d19ea server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:27.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f7679ffb700 1 -- 172.21.15.3:0/770276889 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670011830 con 0x7f767c0cad60 2023-03-03T06:38:27.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f7679ffb700 1 -- 172.21.15.3:0/770276889 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7670011990 con 0x7f767c0cad60 2023-03-03T06:38:27.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f7679ffb700 1 -- 172.21.15.3:0/770276889 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670011d00 con 0x7f767c0cad60 2023-03-03T06:38:27.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f768126a700 1 -- 172.21.15.3:0/770276889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c0cad60 msgr2=0x7f767c0b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:27.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f768126a700 1 --2- 172.21.15.3:0/770276889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c0cad60 0x7f767c0b2540 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7670009850 tx=0x7f7670009b60 comp rx=0 tx=0).stop 2023-03-03T06:38:27.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f768126a700 1 -- 172.21.15.3:0/770276889 shutdown_connections 2023-03-03T06:38:27.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f768126a700 1 --2- 172.21.15.3:0/770276889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c0cad60 0x7f767c0b2540 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:27.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f768126a700 1 -- 172.21.15.3:0/770276889 >> 172.21.15.3:0/770276889 conn(0x7f767c0ccc40 msgr2=0x7f767c0cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:27.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f768126a700 1 -- 172.21.15.3:0/770276889 shutdown_connections 2023-03-03T06:38:27.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.521+0000 7f768126a700 1 -- 172.21.15.3:0/770276889 wait complete. 2023-03-03T06:38:27.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.525+0000 7f768126a700 1 Processor -- start 2023-03-03T06:38:27.536 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.525+0000 7f768126a700 1 -- start start 2023-03-03T06:38:27.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f768126a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06f970 0x7f767c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:27.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f768126a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c0cad60 0x7f767c1512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:27.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f768126a700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f767c158a00 0x7f767c0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:27.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f768126a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0cbb20 con 0x7f767c06f970 2023-03-03T06:38:27.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f768126a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c1517f0 con 0x7f767c0cad60 2023-03-03T06:38:27.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f768126a700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c151a50 con 0x7f767c158a00 2023-03-03T06:38:27.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f767affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06f970 0x7f767c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:27.539 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f767affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06f970 0x7f767c0cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51146/0 (socket says 172.21.15.3:51146) 2023-03-03T06:38:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f767affd700 1 -- 172.21.15.3:0/3147350881 learned_addr learned my addr 172.21.15.3:0/3147350881 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f767bfff700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f767c158a00 0x7f767c0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f767a7fc700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c0cad60 0x7f767c1512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f767affd700 1 -- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f767c158a00 msgr2=0x7f767c0ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f767affd700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f767c158a00 0x7f767c0ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f767affd700 1 -- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c0cad60 msgr2=0x7f767c1512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f767affd700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c0cad60 0x7f767c1512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:27.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f767affd700 1 -- 172.21.15.3:0/3147350881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76700094b0 con 0x7f767c06f970 2023-03-03T06:38:27.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f767bfff700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f767c158a00 0x7f767c0ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:38:27.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f767affd700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06f970 0x7f767c0cb5e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f766c00c730 tx=0x7f766c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:27.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f76637fe700 1 -- 172.21.15.3:0/3147350881 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c0054e0 con 0x7f767c06f970 2023-03-03T06:38:27.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f76637fe700 1 -- 172.21.15.3:0/3147350881 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c0036a0 con 0x7f767c06f970 2023-03-03T06:38:27.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f76637fe700 1 -- 172.21.15.3:0/3147350881 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c003a10 con 0x7f767c06f970 2023-03-03T06:38:27.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f768126a700 1 -- 172.21.15.3:0/3147350881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767c0ce4a0 con 0x7f767c06f970 2023-03-03T06:38:27.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.529+0000 7f768126a700 1 -- 172.21.15.3:0/3147350881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767c06f170 con 0x7f767c06f970 2023-03-03T06:38:27.543 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.533+0000 7f768126a700 1 -- 172.21.15.3:0/3147350881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767c158dd0 con 0x7f767c06f970 2023-03-03T06:38:27.544 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.533+0000 7f76637fe700 1 -- 172.21.15.3:0/3147350881 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f766c019030 con 0x7f767c06f970 2023-03-03T06:38:27.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.537+0000 7f76637fe700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7668065030 0x7f76680674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:27.546 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.537+0000 7f76637fe700 1 -- 172.21.15.3:0/3147350881 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f766c0815d0 con 0x7f767c06f970 2023-03-03T06:38:27.547 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.537+0000 7f767a7fc700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7668065030 0x7f76680674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:27.548 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.537+0000 7f767a7fc700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7668065030 0x7f76680674e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76700090b0 tx=0x7f767001c040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:27.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.541+0000 7f76637fe700 1 -- 172.21.15.3:0/3147350881 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f766c04a400 con 0x7f767c06f970 2023-03-03T06:38:27.746 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.737+0000 7f768126a700 1 -- 172.21.15.3:0/3147350881 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f767c04e180 con 0x7f7668065030 2023-03-03T06:38:27.903 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.893+0000 7f76637fe700 1 -- 172.21.15.3:0/3147350881 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f767c04e180 con 0x7f7668065030 2023-03-03T06:38:27.904 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-03-03T06:38:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.897+0000 7f768126a700 1 -- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7668065030 msgr2=0x7f76680674e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.897+0000 7f768126a700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7668065030 0x7f76680674e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76700090b0 tx=0x7f767001c040 comp rx=0 tx=0).stop 2023-03-03T06:38:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.897+0000 7f768126a700 1 -- 172.21.15.3:0/3147350881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06f970 msgr2=0x7f767c0cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.897+0000 7f768126a700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06f970 0x7f767c0cb5e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f766c00c730 tx=0x7f766c00ca40 comp rx=0 tx=0).stop 2023-03-03T06:38:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.897+0000 7f768126a700 1 -- 172.21.15.3:0/3147350881 shutdown_connections 2023-03-03T06:38:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.897+0000 7f768126a700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f7668065030 0x7f76680674e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.897+0000 7f768126a700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f767c158a00 0x7f767c0ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.897+0000 7f768126a700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f767c0cad60 0x7f767c1512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.897+0000 7f768126a700 1 --2- 172.21.15.3:0/3147350881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f767c06f970 0x7f767c0cb5e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.897+0000 7f768126a700 1 -- 172.21.15.3:0/3147350881 >> 172.21.15.3:0/3147350881 conn(0x7f767c0ccc40 msgr2=0x7f767c0b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.897+0000 7f768126a700 1 -- 172.21.15.3:0/3147350881 shutdown_connections 2023-03-03T06:38:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:27.897+0000 7f768126a700 1 -- 172.21.15.3:0/3147350881 wait complete. 2023-03-03T06:38:27.945 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-03-03T06:38:27.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:27.946 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-03-03T06:38:27.975 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_28_00_UTC 2023-03-03T06:38:27.975 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_29_00_UTC 2023-03-03T06:38:27.975 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_30_00_UTC 2023-03-03T06:38:27.975 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_31_01_UTC 2023-03-03T06:38:27.976 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_32_00_UTC 2023-03-03T06:38:27.976 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_33_00_UTC 2023-03-03T06:38:27.976 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_34_00_UTC 2023-03-03T06:38:27.976 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_35_00_UTC 2023-03-03T06:38:27.976 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_36_01_UTC 2023-03-03T06:38:27.976 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_37_00_UTC 2023-03-03T06:38:27.976 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_38_00_UTC 2023-03-03T06:38:27.978 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-03-03-06_28_00_UTC', 'scheduled-2023-03-03-06_29_00_UTC', 'scheduled-2023-03-03-06_30_00_UTC', 'scheduled-2023-03-03-06_31_01_UTC', 'scheduled-2023-03-03-06_32_00_UTC', 'scheduled-2023-03-03-06_33_00_UTC', 'scheduled-2023-03-03-06_34_00_UTC', 'scheduled-2023-03-03-06_35_00_UTC', 'scheduled-2023-03-03-06_36_01_UTC', 'scheduled-2023-03-03-06_37_00_UTC', 'scheduled-2023-03-03-06_38_00_UTC'] 2023-03-03T06:38:27.978 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir0 2023-03-03T06:38:28.240 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.229+0000 7f4ca9834700 1 Processor -- start 2023-03-03T06:38:28.241 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.229+0000 7f4ca9834700 1 -- start start 2023-03-03T06:38:28.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.233+0000 7f4ca9834700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ca40b6050 0x7f4ca40d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:28.244 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.233+0000 7f4ca9834700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4ca415c960 0x7f4ca40b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:28.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.233+0000 7f4ca9834700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ca40d4040 0x7f4ca406b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:28.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.233+0000 7f4ca9834700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca40d32c0 con 0x7f4ca40b6050 2023-03-03T06:38:28.245 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.233+0000 7f4ca9834700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca40b6960 con 0x7f4ca415c960 2023-03-03T06:38:28.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.233+0000 7f4ca9834700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca40d26b0 con 0x7f4ca40d4040 2023-03-03T06:38:28.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.233+0000 7f4ca2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ca40b6050 0x7f4ca40d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:28.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.233+0000 7f4ca2ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ca40b6050 0x7f4ca40d3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51150/0 (socket says 172.21.15.3:51150) 2023-03-03T06:38:28.246 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.233+0000 7f4ca2ffd700 1 -- 172.21.15.3:0/666037968 learned_addr learned my addr 172.21.15.3:0/666037968 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:28.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.233+0000 7f4ca3fff700 1 --2- 172.21.15.3:0/666037968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ca40d4040 0x7f4ca406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:28.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.233+0000 7f4ca27fc700 1 --2- 172.21.15.3:0/666037968 >> v2:172.21.15.32:3300/0 conn(0x7f4ca415c960 0x7f4ca40b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:28.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca2ffd700 1 -- 172.21.15.3:0/666037968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ca40d4040 msgr2=0x7f4ca406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:28.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca2ffd700 1 --2- 172.21.15.3:0/666037968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ca40d4040 0x7f4ca406b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca2ffd700 1 -- 172.21.15.3:0/666037968 >> v2:172.21.15.32:3300/0 conn(0x7f4ca415c960 msgr2=0x7f4ca40b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:28.247 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca2ffd700 1 --2- 172.21.15.3:0/666037968 >> v2:172.21.15.32:3300/0 conn(0x7f4ca415c960 0x7f4ca40b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca2ffd700 1 -- 172.21.15.3:0/666037968 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca40d2990 con 0x7f4ca40b6050 2023-03-03T06:38:28.248 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca2ffd700 1 --2- 172.21.15.3:0/666037968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ca40b6050 0x7f4ca40d3960 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40d1e80 tx=0x7f4c94009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99890bb0029ad9e8 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:28.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca1ffb700 1 -- 172.21.15.3:0/666037968 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94011830 con 0x7f4ca40b6050 2023-03-03T06:38:28.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca1ffb700 1 -- 172.21.15.3:0/666037968 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c94011990 con 0x7f4ca40b6050 2023-03-03T06:38:28.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca1ffb700 1 -- 172.21.15.3:0/666037968 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94011d20 con 0x7f4ca40b6050 2023-03-03T06:38:28.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca9834700 1 -- 172.21.15.3:0/666037968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ca40b6050 msgr2=0x7f4ca40d3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:28.249 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca9834700 1 --2- 172.21.15.3:0/666037968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ca40b6050 0x7f4ca40d3960 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40d1e80 tx=0x7f4c94009bf0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca9834700 1 -- 172.21.15.3:0/666037968 shutdown_connections 2023-03-03T06:38:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca9834700 1 --2- 172.21.15.3:0/666037968 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ca40d4040 0x7f4ca406b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca9834700 1 --2- 172.21.15.3:0/666037968 >> v2:172.21.15.32:3300/0 conn(0x7f4ca415c960 0x7f4ca40b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca9834700 1 --2- 172.21.15.3:0/666037968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ca40b6050 0x7f4ca40d3960 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca9834700 1 -- 172.21.15.3:0/666037968 >> 172.21.15.3:0/666037968 conn(0x7f4ca406a9b0 msgr2=0x7f4ca40d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca9834700 1 -- 172.21.15.3:0/666037968 shutdown_connections 2023-03-03T06:38:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.237+0000 7f4ca9834700 1 -- 172.21.15.3:0/666037968 wait complete. 2023-03-03T06:38:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.241+0000 7f4ca9834700 1 Processor -- start 2023-03-03T06:38:28.252 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.241+0000 7f4ca9834700 1 -- start start 2023-03-03T06:38:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.241+0000 7f4ca9834700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ca40b6050 0x7f4ca4160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.241+0000 7f4ca9834700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ca40d4040 0x7f4ca40c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.241+0000 7f4ca9834700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca415c960 0x7f4ca41f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.241+0000 7f4ca9834700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca40cb1b0 con 0x7f4ca40d4040 2023-03-03T06:38:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.241+0000 7f4ca9834700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca4160ca0 con 0x7f4ca415c960 2023-03-03T06:38:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.241+0000 7f4ca9834700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca40ca810 con 0x7f4ca40b6050 2023-03-03T06:38:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4ca27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ca40d4040 0x7f4ca40c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4ca27fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ca40d4040 0x7f4ca40c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51166/0 (socket says 172.21.15.3:51166) 2023-03-03T06:38:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4ca27fc700 1 -- 172.21.15.3:0/1918893868 learned_addr learned my addr 172.21.15.3:0/1918893868 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4ca2ffd700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ca40b6050 0x7f4ca4160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4ca3fff700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca415c960 0x7f4ca41f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4ca27fc700 1 -- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ca40b6050 msgr2=0x7f4ca4160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4ca27fc700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ca40b6050 0x7f4ca4160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4ca27fc700 1 -- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca415c960 msgr2=0x7f4ca41f95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:28.256 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4ca27fc700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca415c960 0x7f4ca41f95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4ca27fc700 1 -- 172.21.15.3:0/1918893868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c94009570 con 0x7f4ca40d4040 2023-03-03T06:38:28.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4ca27fc700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ca40d4040 0x7f4ca40c82d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009fd0 tx=0x7f4c9800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:28.257 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4c8bfff700 1 -- 172.21.15.3:0/1918893868 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980095a0 con 0x7f4ca40d4040 2023-03-03T06:38:28.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4c8bfff700 1 -- 172.21.15.3:0/1918893868 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c98013680 con 0x7f4ca40d4040 2023-03-03T06:38:28.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4c8bfff700 1 -- 172.21.15.3:0/1918893868 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98013a10 con 0x7f4ca40d4040 2023-03-03T06:38:28.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4ca9834700 1 -- 172.21.15.3:0/1918893868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca40d2f50 con 0x7f4ca40d4040 2023-03-03T06:38:28.258 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.245+0000 7f4ca9834700 1 -- 172.21.15.3:0/1918893868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca40d2c70 con 0x7f4ca40d4040 2023-03-03T06:38:28.259 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.249+0000 7f4ca9834700 1 -- 172.21.15.3:0/1918893868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca40cf7f0 con 0x7f4ca40d4040 2023-03-03T06:38:28.261 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.249+0000 7f4c8bfff700 1 -- 172.21.15.3:0/1918893868 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f4c98019030 con 0x7f4ca40d4040 2023-03-03T06:38:28.262 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.253+0000 7f4c8bfff700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c90065000 0x7f4c900674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:28.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.253+0000 7f4c8bfff700 1 -- 172.21.15.3:0/1918893868 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f4c98082190 con 0x7f4ca40d4040 2023-03-03T06:38:28.263 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.253+0000 7f4ca2ffd700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c90065000 0x7f4c900674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:28.265 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.253+0000 7f4ca2ffd700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c90065000 0x7f4c900674b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009860 tx=0x7f4c94008030 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:28.268 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.257+0000 7f4c8bfff700 1 -- 172.21.15.3:0/1918893868 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4c9804fe90 con 0x7f4ca40d4040 2023-03-03T06:38:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.453+0000 7f4ca9834700 1 -- 172.21.15.3:0/1918893868 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ca40cb420 con 0x7f4c90065000 2023-03-03T06:38:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.473+0000 7f4c8bfff700 1 -- 172.21.15.3:0/1918893868 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f4ca40cb420 con 0x7f4c90065000 2023-03-03T06:38:28.484 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:38:28.484 INFO:teuthology.orchestra.run.smithi003.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:15", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:28.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.477+0000 7f4ca9834700 1 -- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c90065000 msgr2=0x7f4c900674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:28.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.477+0000 7f4ca9834700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c90065000 0x7f4c900674b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009860 tx=0x7f4c94008030 comp rx=0 tx=0).stop 2023-03-03T06:38:28.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.477+0000 7f4ca9834700 1 -- 172.21.15.3:0/1918893868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ca40d4040 msgr2=0x7f4ca40c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:28.490 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.477+0000 7f4ca9834700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ca40d4040 0x7f4ca40c82d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009fd0 tx=0x7f4c9800eb20 comp rx=0 tx=0).stop 2023-03-03T06:38:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.481+0000 7f4ca9834700 1 -- 172.21.15.3:0/1918893868 shutdown_connections 2023-03-03T06:38:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.481+0000 7f4ca9834700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f4c90065000 0x7f4c900674b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.481+0000 7f4ca9834700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca415c960 0x7f4ca41f95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.481+0000 7f4ca9834700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ca40d4040 0x7f4ca40c82d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.481+0000 7f4ca9834700 1 --2- 172.21.15.3:0/1918893868 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f4ca40b6050 0x7f4ca4160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.491 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.481+0000 7f4ca9834700 1 -- 172.21.15.3:0/1918893868 >> 172.21.15.3:0/1918893868 conn(0x7f4ca406a9b0 msgr2=0x7f4ca40705d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:28.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.481+0000 7f4ca9834700 1 -- 172.21.15.3:0/1918893868 shutdown_connections 2023-03-03T06:38:28.492 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.481+0000 7f4ca9834700 1 -- 172.21.15.3:0/1918893868 wait complete. 2023-03-03T06:38:28.522 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:15", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:28.522 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-03-03T00:00:00', 'created': '2023-03-03T06:27:15', 'first': '2023-03-03T06:28:00', 'last': '2023-03-03T06:38:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-03-03T06:38:28.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:28.523 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-03-03T06:38:28.551 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_28_00_UTC 2023-03-03T06:38:28.551 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_29_00_UTC 2023-03-03T06:38:28.551 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_30_00_UTC 2023-03-03T06:38:28.551 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_31_00_UTC 2023-03-03T06:38:28.552 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_32_00_UTC 2023-03-03T06:38:28.552 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_33_01_UTC 2023-03-03T06:38:28.552 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_34_00_UTC 2023-03-03T06:38:28.552 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_35_00_UTC 2023-03-03T06:38:28.552 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_36_00_UTC 2023-03-03T06:38:28.552 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_37_01_UTC 2023-03-03T06:38:28.552 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_38_01_UTC 2023-03-03T06:38:28.553 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-03-03-06_28_00_UTC', 'scheduled-2023-03-03-06_29_00_UTC', 'scheduled-2023-03-03-06_30_00_UTC', 'scheduled-2023-03-03-06_31_00_UTC', 'scheduled-2023-03-03-06_32_00_UTC', 'scheduled-2023-03-03-06_33_01_UTC', 'scheduled-2023-03-03-06_34_00_UTC', 'scheduled-2023-03-03-06_35_00_UTC', 'scheduled-2023-03-03-06_36_00_UTC', 'scheduled-2023-03-03-06_37_01_UTC', 'scheduled-2023-03-03-06_38_01_UTC'] 2023-03-03T06:38:28.554 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir1 2023-03-03T06:38:28.813 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.801+0000 7fc18e37f700 1 Processor -- start 2023-03-03T06:38:28.815 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.805+0000 7fc18e37f700 1 -- start start 2023-03-03T06:38:28.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18e37f700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc1880caab0 0x7fc1880c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:28.818 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18e37f700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1880b1850 0x7fc1880b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18e37f700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18806d750 con 0x7fc1880b07c0 2023-03-03T06:38:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18d37d700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc1880b07c0 0x7fc1880b0b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36876/0 (socket says 172.21.15.3:36876) 2023-03-03T06:38:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18d37d700 1 -- 172.21.15.3:0/1663147796 learned_addr learned my addr 172.21.15.3:0/1663147796 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:38:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18e37f700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc18815a640 con 0x7fc1880caab0 2023-03-03T06:38:28.819 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18e37f700 1 -- 172.21.15.3:0/1663147796 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc18806cb40 con 0x7fc1880b1850 2023-03-03T06:38:28.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18cb7c700 1 --2- 172.21.15.3:0/1663147796 >> v2:172.21.15.3:3300/0 conn(0x7fc1880caab0 0x7fc1880c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:28.820 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18db7e700 1 --2- 172.21.15.3:0/1663147796 >> v2:172.21.15.32:3300/0 conn(0x7fc1880b1850 0x7fc1880b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:28.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc17ffff700 1 -- 172.21.15.3:0/1663147796 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134896711 0 0) 0x7fc18806d750 con 0x7fc1880b07c0 2023-03-03T06:38:28.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc17ffff700 1 -- 172.21.15.3:0/1663147796 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1740035d0 con 0x7fc1880b07c0 2023-03-03T06:38:28.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18cb7c700 1 -- 172.21.15.3:0/1663147796 >> v2:172.21.15.32:3300/0 conn(0x7fc1880b1850 msgr2=0x7fc1880b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:28.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18cb7c700 1 --2- 172.21.15.3:0/1663147796 >> v2:172.21.15.32:3300/0 conn(0x7fc1880b1850 0x7fc1880b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18cb7c700 1 -- 172.21.15.3:0/1663147796 >> v1:172.21.15.3:6789/0 conn(0x7fc1880b07c0 legacy=0x7fc1880b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:28.821 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18cb7c700 1 -- 172.21.15.3:0/1663147796 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18806ce20 con 0x7fc1880caab0 2023-03-03T06:38:28.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc18cb7c700 1 --2- 172.21.15.3:0/1663147796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1880caab0 0x7fc1880c8e00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc1880c8d90 tx=0x7fc178009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f09a3b70790ab72 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:28.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc17ffff700 1 -- 172.21.15.3:0/1663147796 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178011830 con 0x7fc1880caab0 2023-03-03T06:38:28.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc17ffff700 1 -- 172.21.15.3:0/1663147796 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc178011990 con 0x7fc1880caab0 2023-03-03T06:38:28.822 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.809+0000 7fc17ffff700 1 -- 172.21.15.3:0/1663147796 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178011d20 con 0x7fc1880caab0 2023-03-03T06:38:28.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.813+0000 7fc18e37f700 1 -- 172.21.15.3:0/1663147796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1880caab0 msgr2=0x7fc1880c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:28.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.813+0000 7fc18e37f700 1 --2- 172.21.15.3:0/1663147796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1880caab0 0x7fc1880c8e00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc1880c8d90 tx=0x7fc178009b30 comp rx=0 tx=0).stop 2023-03-03T06:38:28.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.813+0000 7fc18e37f700 1 -- 172.21.15.3:0/1663147796 shutdown_connections 2023-03-03T06:38:28.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.813+0000 7fc18e37f700 1 --2- 172.21.15.3:0/1663147796 >> v2:172.21.15.32:3300/0 conn(0x7fc1880b1850 0x7fc1880b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.823 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.813+0000 7fc18e37f700 1 --2- 172.21.15.3:0/1663147796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1880caab0 0x7fc1880c8e00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.813+0000 7fc18e37f700 1 -- 172.21.15.3:0/1663147796 >> 172.21.15.3:0/1663147796 conn(0x7fc1880607c0 msgr2=0x7fc188060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:28.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.813+0000 7fc18e37f700 1 -- 172.21.15.3:0/1663147796 shutdown_connections 2023-03-03T06:38:28.824 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.813+0000 7fc18e37f700 1 -- 172.21.15.3:0/1663147796 wait complete. 2023-03-03T06:38:28.827 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.817+0000 7fc18e37f700 1 Processor -- start 2023-03-03T06:38:28.828 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.817+0000 7fc18e37f700 1 -- start start 2023-03-03T06:38:28.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.817+0000 7fc18e37f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1880b07c0 0x7fc188148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:28.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.817+0000 7fc18e37f700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc1880b1850 0x7fc1881f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:28.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.817+0000 7fc18e37f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1880caab0 0x7fc1881f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:28.829 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.817+0000 7fc18e37f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880c27a0 con 0x7fc1880caab0 2023-03-03T06:38:28.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.817+0000 7fc18e37f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc188149250 con 0x7fc1880b07c0 2023-03-03T06:38:28.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.817+0000 7fc18e37f700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880b4120 con 0x7fc1880b1850 2023-03-03T06:38:28.830 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18db7e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1880caab0 0x7fc1881f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18db7e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1880caab0 0x7fc1881f5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51188/0 (socket says 172.21.15.3:51188) 2023-03-03T06:38:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18db7e700 1 -- 172.21.15.3:0/1451838292 learned_addr learned my addr 172.21.15.3:0/1451838292 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18d37d700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1880b07c0 0x7fc188148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18cb7c700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc1880b1850 0x7fc1881f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:28.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18db7e700 1 -- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc1880b1850 msgr2=0x7fc1881f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:28.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18db7e700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc1880b1850 0x7fc1881f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18db7e700 1 -- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1880b07c0 msgr2=0x7fc188148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:28.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18db7e700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1880b07c0 0x7fc188148d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:28.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18db7e700 1 -- 172.21.15.3:0/1451838292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780094b0 con 0x7fc1880caab0 2023-03-03T06:38:28.832 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18cb7c700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc1880b1850 0x7fc1881f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:38:28.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18d37d700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1880b07c0 0x7fc188148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:38:28.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18db7e700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1880caab0 0x7fc1881f5570 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc18000efc0 tx=0x7fc18000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:28.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc17dffb700 1 -- 172.21.15.3:0/1451838292 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18000cd40 con 0x7fc1880caab0 2023-03-03T06:38:28.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc17dffb700 1 -- 172.21.15.3:0/1451838292 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18000f040 con 0x7fc1880caab0 2023-03-03T06:38:28.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18e37f700 1 -- 172.21.15.3:0/1451838292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18806d3e0 con 0x7fc1880caab0 2023-03-03T06:38:28.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc17dffb700 1 -- 172.21.15.3:0/1451838292 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800146c0 con 0x7fc1880caab0 2023-03-03T06:38:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.821+0000 7fc18e37f700 1 -- 172.21.15.3:0/1451838292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18806d100 con 0x7fc1880caab0 2023-03-03T06:38:28.835 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.825+0000 7fc18e37f700 1 -- 172.21.15.3:0/1451838292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1880adf90 con 0x7fc1880caab0 2023-03-03T06:38:28.836 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.825+0000 7fc17dffb700 1 -- 172.21.15.3:0/1451838292 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7fc180014820 con 0x7fc1880caab0 2023-03-03T06:38:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.829+0000 7fc17dffb700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc16c065000 0x7fc16c0674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.829+0000 7fc17dffb700 1 -- 172.21.15.3:0/1451838292 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fc180082f70 con 0x7fc1880caab0 2023-03-03T06:38:28.840 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.829+0000 7fc18d37d700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc16c065000 0x7fc16c0674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:28.841 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.829+0000 7fc18d37d700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc16c065000 0x7fc16c0674b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc1840011e0 tx=0x7fc184009040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:28.833+0000 7fc17dffb700 1 -- 172.21.15.3:0/1451838292 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc180050cd0 con 0x7fc1880caab0 2023-03-03T06:38:29.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.033+0000 7fc18e37f700 1 -- 172.21.15.3:0/1451838292 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc188073e90 con 0x7fc16c065000 2023-03-03T06:38:29.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.049+0000 7fc17dffb700 1 -- 172.21.15.3:0/1451838292 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fc188073e90 con 0x7fc16c065000 2023-03-03T06:38:29.062 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:38:29.062 INFO:teuthology.orchestra.run.smithi003.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:15", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.057+0000 7fc18e37f700 1 -- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc16c065000 msgr2=0x7fc16c0674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.057+0000 7fc18e37f700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc16c065000 0x7fc16c0674b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc1840011e0 tx=0x7fc184009040 comp rx=0 tx=0).stop 2023-03-03T06:38:29.067 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.057+0000 7fc18e37f700 1 -- 172.21.15.3:0/1451838292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1880caab0 msgr2=0x7fc1881f5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:29.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.057+0000 7fc18e37f700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1880caab0 0x7fc1881f5570 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc18000efc0 tx=0x7fc18000c3a0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.057+0000 7fc18e37f700 1 -- 172.21.15.3:0/1451838292 shutdown_connections 2023-03-03T06:38:29.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.057+0000 7fc18e37f700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fc16c065000 0x7fc16c0674b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.057+0000 7fc18e37f700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1880caab0 0x7fc1881f5570 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.057+0000 7fc18e37f700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fc1880b1850 0x7fc1881f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.057+0000 7fc18e37f700 1 --2- 172.21.15.3:0/1451838292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1880b07c0 0x7fc188148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.057+0000 7fc18e37f700 1 -- 172.21.15.3:0/1451838292 >> 172.21.15.3:0/1451838292 conn(0x7fc1880607c0 msgr2=0x7fc1880cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:29.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.057+0000 7fc18e37f700 1 -- 172.21.15.3:0/1451838292 shutdown_connections 2023-03-03T06:38:29.069 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.057+0000 7fc18e37f700 1 -- 172.21.15.3:0/1451838292 wait complete. 2023-03-03T06:38:29.100 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:15", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:29.101 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-03-03T00:00:00', 'created': '2023-03-03T06:27:15', 'first': '2023-03-03T06:28:00', 'last': '2023-03-03T06:38:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-03-03T06:38:29.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:29.102 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-03-03T06:38:29.126 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_28_00_UTC 2023-03-03T06:38:29.126 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_29_00_UTC 2023-03-03T06:38:29.127 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_30_00_UTC 2023-03-03T06:38:29.127 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_31_00_UTC 2023-03-03T06:38:29.127 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_32_00_UTC 2023-03-03T06:38:29.127 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_33_00_UTC 2023-03-03T06:38:29.127 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_34_00_UTC 2023-03-03T06:38:29.127 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_35_00_UTC 2023-03-03T06:38:29.127 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_36_00_UTC 2023-03-03T06:38:29.128 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_37_00_UTC 2023-03-03T06:38:29.128 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_38_00_UTC 2023-03-03T06:38:29.128 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-03-03-06_28_00_UTC', 'scheduled-2023-03-03-06_29_00_UTC', 'scheduled-2023-03-03-06_30_00_UTC', 'scheduled-2023-03-03-06_31_00_UTC', 'scheduled-2023-03-03-06_32_00_UTC', 'scheduled-2023-03-03-06_33_00_UTC', 'scheduled-2023-03-03-06_34_00_UTC', 'scheduled-2023-03-03-06_35_00_UTC', 'scheduled-2023-03-03-06_36_00_UTC', 'scheduled-2023-03-03-06_37_00_UTC', 'scheduled-2023-03-03-06_38_00_UTC'] 2023-03-03T06:38:29.129 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir2 2023-03-03T06:38:29.391 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.381+0000 7f4625549700 1 Processor -- start 2023-03-03T06:38:29.392 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.381+0000 7f4625549700 1 -- start start 2023-03-03T06:38:29.393 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.381+0000 7f4625549700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46201615f0 0x7f46201619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:29.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.381+0000 7f4625549700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f46200b1c40 0x7f46200b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:29.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f4625549700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4620164410 con 0x7f4620165140 2023-03-03T06:38:29.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f4625549700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46200b2620 con 0x7f46200b1c40 2023-03-03T06:38:29.394 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f4625549700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46200cb300 con 0x7f46201615f0 2023-03-03T06:38:29.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f461effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46201615f0 0x7f46201619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:29.395 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f461effd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46201615f0 0x7f46201619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.3:57822/0 (socket says 172.21.15.3:57822) 2023-03-03T06:38:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f461effd700 1 -- 172.21.15.3:0/2829133633 learned_addr learned my addr 172.21.15.3:0/2829133633 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f461ffff700 1 --2- 172.21.15.3:0/2829133633 >> v2:172.21.15.3:3300/0 conn(0x7f46200b1c40 0x7f46200b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:29.396 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f461effd700 1 -- 172.21.15.3:0/2829133633 >> v2:172.21.15.3:3300/0 conn(0x7f46200b1c40 msgr2=0x7f46200b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:29.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f461effd700 1 --2- 172.21.15.3:0/2829133633 >> v2:172.21.15.3:3300/0 conn(0x7f46200b1c40 0x7f46200b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f461effd700 1 -- 172.21.15.3:0/2829133633 >> v1:172.21.15.3:6789/0 conn(0x7f4620165140 legacy=0x7f46200ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:29.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f461effd700 1 -- 172.21.15.3:0/2829133633 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4620164660 con 0x7f46201615f0 2023-03-03T06:38:29.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f461ffff700 1 --2- 172.21.15.3:0/2829133633 >> v2:172.21.15.3:3300/0 conn(0x7f46200b1c40 0x7f46200b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:38:29.397 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f461dffb700 1 -- 172.21.15.3:0/2829133633 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050696765 0 0) 0x7f4620164410 con 0x7f4620165140 2023-03-03T06:38:29.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f461effd700 1 --2- 172.21.15.3:0/2829133633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46201615f0 0x7f46201619c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f462006af40 tx=0x7f4610009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0221c67be6a2cc server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:29.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.385+0000 7f461dffb700 1 -- 172.21.15.3:0/2829133633 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610005e50 con 0x7f46201615f0 2023-03-03T06:38:29.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.389+0000 7f461dffb700 1 -- 172.21.15.3:0/2829133633 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4610006030 con 0x7f46201615f0 2023-03-03T06:38:29.398 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.389+0000 7f461dffb700 1 -- 172.21.15.3:0/2829133633 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100063c0 con 0x7f46201615f0 2023-03-03T06:38:29.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.389+0000 7f4625549700 1 -- 172.21.15.3:0/2829133633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46201615f0 msgr2=0x7f46201619c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:29.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.389+0000 7f4625549700 1 --2- 172.21.15.3:0/2829133633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46201615f0 0x7f46201619c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f462006af40 tx=0x7f4610009bf0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.389+0000 7f4625549700 1 -- 172.21.15.3:0/2829133633 shutdown_connections 2023-03-03T06:38:29.399 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.389+0000 7f4625549700 1 --2- 172.21.15.3:0/2829133633 >> v2:172.21.15.3:3300/0 conn(0x7f46200b1c40 0x7f46200b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.389+0000 7f4625549700 1 --2- 172.21.15.3:0/2829133633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46201615f0 0x7f46201619c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.389+0000 7f4625549700 1 -- 172.21.15.3:0/2829133633 >> 172.21.15.3:0/2829133633 conn(0x7f46200ccc40 msgr2=0x7f46200cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:29.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.389+0000 7f4625549700 1 -- 172.21.15.3:0/2829133633 shutdown_connections 2023-03-03T06:38:29.400 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.389+0000 7f4625549700 1 -- 172.21.15.3:0/2829133633 wait complete. 2023-03-03T06:38:29.403 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.393+0000 7f4625549700 1 Processor -- start 2023-03-03T06:38:29.404 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.393+0000 7f4625549700 1 -- start start 2023-03-03T06:38:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.393+0000 7f4625549700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46200b1c40 0x7f46201596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.393+0000 7f4625549700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46201615f0 0x7f46200c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.393+0000 7f4625549700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4620165140 0x7f46200c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:29.405 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.393+0000 7f4625549700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46200b5350 con 0x7f4620165140 2023-03-03T06:38:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.393+0000 7f4625549700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46200b5630 con 0x7f46200b1c40 2023-03-03T06:38:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.393+0000 7f4625549700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46200b5910 con 0x7f46201615f0 2023-03-03T06:38:29.406 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.393+0000 7f461ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4620165140 0x7f46200c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.393+0000 7f461ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4620165140 0x7f46200c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51204/0 (socket says 172.21.15.3:51204) 2023-03-03T06:38:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.393+0000 7f461ffff700 1 -- 172.21.15.3:0/442563602 learned_addr learned my addr 172.21.15.3:0/442563602 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.393+0000 7f461effd700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46200b1c40 0x7f46201596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:29.407 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.397+0000 7f461e7fc700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46201615f0 0x7f46200c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.397+0000 7f461ffff700 1 -- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46201615f0 msgr2=0x7f46200c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.397+0000 7f461ffff700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46201615f0 0x7f46200c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.397+0000 7f461ffff700 1 -- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46200b1c40 msgr2=0x7f46201596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.397+0000 7f461ffff700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46200b1c40 0x7f46201596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.408 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.397+0000 7f461ffff700 1 -- 172.21.15.3:0/442563602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4618009a70 con 0x7f4620165140 2023-03-03T06:38:29.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.397+0000 7f461ffff700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4620165140 0x7f46200c6180 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f461800efc0 tx=0x7f461800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:29.409 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.397+0000 7f46077fe700 1 -- 172.21.15.3:0/442563602 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461800ca90 con 0x7f4620165140 2023-03-03T06:38:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.397+0000 7f46077fe700 1 -- 172.21.15.3:0/442563602 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461800f040 con 0x7f4620165140 2023-03-03T06:38:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.397+0000 7f4625549700 1 -- 172.21.15.3:0/442563602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4610009520 con 0x7f4620165140 2023-03-03T06:38:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.397+0000 7f46077fe700 1 -- 172.21.15.3:0/442563602 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461800ce30 con 0x7f4620165140 2023-03-03T06:38:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.397+0000 7f4625549700 1 -- 172.21.15.3:0/442563602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4620164c20 con 0x7f4620165140 2023-03-03T06:38:29.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.397+0000 7f4625549700 1 -- 172.21.15.3:0/442563602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4620164940 con 0x7f4620165140 2023-03-03T06:38:29.412 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.401+0000 7f46077fe700 1 -- 172.21.15.3:0/442563602 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f4618014480 con 0x7f4620165140 2023-03-03T06:38:29.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.405+0000 7f46077fe700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f460c065000 0x7f460c0674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:29.415 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.405+0000 7f46077fe700 1 -- 172.21.15.3:0/442563602 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f4618082b10 con 0x7f4620165140 2023-03-03T06:38:29.416 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.405+0000 7f461effd700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f460c065000 0x7f460c0674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:29.417 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.405+0000 7f461effd700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f460c065000 0x7f460c0674b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4610013040 tx=0x7f4610008070 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:29.420 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.409+0000 7f46077fe700 1 -- 172.21.15.3:0/442563602 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4618050810 con 0x7f4620165140 2023-03-03T06:38:29.566 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:38:29.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:38:29.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.609+0000 7f4625549700 1 -- 172.21.15.3:0/442563602 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46200caf20 con 0x7f460c065000 2023-03-03T06:38:29.638 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.625+0000 7f46077fe700 1 -- 172.21.15.3:0/442563602 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f46200caf20 con 0x7f460c065000 2023-03-03T06:38:29.638 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:38:29.638 INFO:teuthology.orchestra.run.smithi003.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:16", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.633+0000 7f4625549700 1 -- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f460c065000 msgr2=0x7f460c0674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.633+0000 7f4625549700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f460c065000 0x7f460c0674b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4610013040 tx=0x7f4610008070 comp rx=0 tx=0).stop 2023-03-03T06:38:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.633+0000 7f4625549700 1 -- 172.21.15.3:0/442563602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4620165140 msgr2=0x7f46200c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.633+0000 7f4625549700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4620165140 0x7f46200c6180 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f461800efc0 tx=0x7f461800c3a0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.633+0000 7f4625549700 1 -- 172.21.15.3:0/442563602 shutdown_connections 2023-03-03T06:38:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.633+0000 7f4625549700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f460c065000 0x7f460c0674b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.633+0000 7f4625549700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4620165140 0x7f46200c6180 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.633+0000 7f4625549700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f46201615f0 0x7f46200c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.633+0000 7f4625549700 1 --2- 172.21.15.3:0/442563602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46200b1c40 0x7f46201596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.633+0000 7f4625549700 1 -- 172.21.15.3:0/442563602 >> 172.21.15.3:0/442563602 conn(0x7f46200ccc40 msgr2=0x7f46200cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.633+0000 7f4625549700 1 -- 172.21.15.3:0/442563602 shutdown_connections 2023-03-03T06:38:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:29.633+0000 7f4625549700 1 -- 172.21.15.3:0/442563602 wait complete. 2023-03-03T06:38:29.674 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:16", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:29.674 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-03-03T00:00:00', 'created': '2023-03-03T06:27:16', 'first': '2023-03-03T06:28:00', 'last': '2023-03-03T06:38:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-03-03T06:38:29.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:29.675 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-03-03T06:38:29.751 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_28_01_UTC 2023-03-03T06:38:29.752 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_29_00_UTC 2023-03-03T06:38:29.752 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_30_01_UTC 2023-03-03T06:38:29.752 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_31_00_UTC 2023-03-03T06:38:29.752 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_32_00_UTC 2023-03-03T06:38:29.752 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_33_01_UTC 2023-03-03T06:38:29.752 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_34_00_UTC 2023-03-03T06:38:29.752 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_35_00_UTC 2023-03-03T06:38:29.753 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_36_01_UTC 2023-03-03T06:38:29.753 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_37_01_UTC 2023-03-03T06:38:29.753 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_38_00_UTC 2023-03-03T06:38:29.754 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-03-03-06_28_01_UTC', 'scheduled-2023-03-03-06_29_00_UTC', 'scheduled-2023-03-03-06_30_01_UTC', 'scheduled-2023-03-03-06_31_00_UTC', 'scheduled-2023-03-03-06_32_00_UTC', 'scheduled-2023-03-03-06_33_01_UTC', 'scheduled-2023-03-03-06_34_00_UTC', 'scheduled-2023-03-03-06_35_00_UTC', 'scheduled-2023-03-03-06_36_01_UTC', 'scheduled-2023-03-03-06_37_01_UTC', 'scheduled-2023-03-03-06_38_00_UTC'] 2023-03-03T06:38:29.754 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir3 2023-03-03T06:38:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.009+0000 7f0f1759e700 1 Processor -- start 2023-03-03T06:38:30.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.009+0000 7f0f1759e700 1 -- start start 2023-03-03T06:38:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.013+0000 7f0f1759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f1815d6e0 0x7f0f180cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.013+0000 7f0f1759e700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0f180cfd80 0x7f0f180d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.013+0000 7f0f1759e700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f180cf3b0 con 0x7f0f18156ba0 2023-03-03T06:38:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.013+0000 7f0f1759e700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f18156f70 con 0x7f0f180cfd80 2023-03-03T06:38:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.013+0000 7f0f1759e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f180cd320 con 0x7f0f1815d6e0 2023-03-03T06:38:30.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.013+0000 7f0f15d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f1815d6e0 0x7f0f180cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:30.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.013+0000 7f0f16d9d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0f180cfd80 0x7f0f180d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:30.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.017+0000 7f0f1659c700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0f18156ba0 0x7f0f18071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:60686/0 (socket says 172.21.15.3:60686) 2023-03-03T06:38:30.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.017+0000 7f0f1659c700 1 -- 172.21.15.3:0/1292887236 learned_addr learned my addr 172.21.15.3:0/1292887236 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:38:30.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.017+0000 7f0f15d9b700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f1815d6e0 0x7f0f180cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:47498/0 (socket says 172.21.15.3:47498) 2023-03-03T06:38:30.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.017+0000 7f0f16d9d700 1 -- 172.21.15.3:0/1292887236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f1815d6e0 msgr2=0x7f0f180cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:30.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.017+0000 7f0f16d9d700 1 --2- 172.21.15.3:0/1292887236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f1815d6e0 0x7f0f180cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.017+0000 7f0f16d9d700 1 -- 172.21.15.3:0/1292887236 >> v1:172.21.15.32:6789/0 conn(0x7f0f18156ba0 legacy=0x7f0f18071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:30.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.017+0000 7f0f16d9d700 1 -- 172.21.15.3:0/1292887236 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f180cd600 con 0x7f0f180cfd80 2023-03-03T06:38:30.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.017+0000 7f0f15d9b700 1 --2- 172.21.15.3:0/1292887236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f1815d6e0 0x7f0f180cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-03T06:38:30.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.017+0000 7f0f1559a700 1 -- 172.21.15.3:0/1292887236 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332772969 0 0) 0x7f0f180cf3b0 con 0x7f0f18156ba0 2023-03-03T06:38:30.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.017+0000 7f0f16d9d700 1 --2- 172.21.15.3:0/1292887236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f180cfd80 0x7f0f180d0b30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009d90 tx=0x7f0f1000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29d3bd8b7133db07 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:30.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.017+0000 7f0f1559a700 1 -- 172.21.15.3:0/1292887236 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1000e070 con 0x7f0f180cfd80 2023-03-03T06:38:30.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.017+0000 7f0f1559a700 1 -- 172.21.15.3:0/1292887236 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1000bb20 con 0x7f0f180cfd80 2023-03-03T06:38:30.029 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.017+0000 7f0f1559a700 1 -- 172.21.15.3:0/1292887236 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10012430 con 0x7f0f180cfd80 2023-03-03T06:38:30.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.021+0000 7f0f1759e700 1 -- 172.21.15.3:0/1292887236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f180cfd80 msgr2=0x7f0f180d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:30.030 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.021+0000 7f0f1759e700 1 --2- 172.21.15.3:0/1292887236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f180cfd80 0x7f0f180d0b30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009d90 tx=0x7f0f1000b330 comp rx=0 tx=0).stop 2023-03-03T06:38:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.021+0000 7f0f1759e700 1 -- 172.21.15.3:0/1292887236 shutdown_connections 2023-03-03T06:38:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.021+0000 7f0f1759e700 1 --2- 172.21.15.3:0/1292887236 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f180cfd80 0x7f0f180d0b30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.021+0000 7f0f1759e700 1 --2- 172.21.15.3:0/1292887236 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f1815d6e0 0x7f0f180cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.021+0000 7f0f1759e700 1 -- 172.21.15.3:0/1292887236 >> 172.21.15.3:0/1292887236 conn(0x7f0f1806e5c0 msgr2=0x7f0f1806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.021+0000 7f0f1759e700 1 -- 172.21.15.3:0/1292887236 shutdown_connections 2023-03-03T06:38:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.021+0000 7f0f1759e700 1 -- 172.21.15.3:0/1292887236 wait complete. 2023-03-03T06:38:30.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.025+0000 7f0f1759e700 1 Processor -- start 2023-03-03T06:38:30.035 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.025+0000 7f0f1759e700 1 -- start start 2023-03-03T06:38:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.025+0000 7f0f1759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f180cfd80 0x7f0f1814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.025+0000 7f0f1759e700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f18156ba0 0x7f0f180b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.025+0000 7f0f1759e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1815d6e0 0x7f0f180ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.025+0000 7f0f1759e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f180b68b0 con 0x7f0f1815d6e0 2023-03-03T06:38:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.025+0000 7f0f1759e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f180b62f0 con 0x7f0f180cfd80 2023-03-03T06:38:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.025+0000 7f0f1759e700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f180b65d0 con 0x7f0f18156ba0 2023-03-03T06:38:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.025+0000 7f0f16d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1815d6e0 0x7f0f180ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.025+0000 7f0f16d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1815d6e0 0x7f0f180ca6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51212/0 (socket says 172.21.15.3:51212) 2023-03-03T06:38:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.025+0000 7f0f16d9d700 1 -- 172.21.15.3:0/2069256704 learned_addr learned my addr 172.21.15.3:0/2069256704 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.025+0000 7f0f1659c700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f180cfd80 0x7f0f1814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:30.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.029+0000 7f0f15d9b700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f18156ba0 0x7f0f180b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:30.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.029+0000 7f0f16d9d700 1 -- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f18156ba0 msgr2=0x7f0f180b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:30.039 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.029+0000 7f0f16d9d700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f18156ba0 0x7f0f180b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.029+0000 7f0f16d9d700 1 -- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f180cfd80 msgr2=0x7f0f1814da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.029+0000 7f0f16d9d700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f180cfd80 0x7f0f1814da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.029+0000 7f0f16d9d700 1 -- 172.21.15.3:0/2069256704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0094b0 con 0x7f0f1815d6e0 2023-03-03T06:38:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.029+0000 7f0f16d9d700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1815d6e0 0x7f0f180ca6c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f1806f190 tx=0x7f0f10004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.029+0000 7f0f06ffd700 1 -- 172.21.15.3:0/2069256704 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10012430 con 0x7f0f1815d6e0 2023-03-03T06:38:30.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.029+0000 7f0f06ffd700 1 -- 172.21.15.3:0/2069256704 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f10020070 con 0x7f0f1815d6e0 2023-03-03T06:38:30.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.029+0000 7f0f06ffd700 1 -- 172.21.15.3:0/2069256704 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10004280 con 0x7f0f1815d6e0 2023-03-03T06:38:30.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.029+0000 7f0f1759e700 1 -- 172.21.15.3:0/2069256704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f10009a70 con 0x7f0f1815d6e0 2023-03-03T06:38:30.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.029+0000 7f0f1759e700 1 -- 172.21.15.3:0/2069256704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f180cef40 con 0x7f0f1815d6e0 2023-03-03T06:38:30.042 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.033+0000 7f0f1759e700 1 -- 172.21.15.3:0/2069256704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f180cd8e0 con 0x7f0f1815d6e0 2023-03-03T06:38:30.043 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.033+0000 7f0f06ffd700 1 -- 172.21.15.3:0/2069256704 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f0f1002e070 con 0x7f0f1815d6e0 2023-03-03T06:38:30.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.033+0000 7f0f06ffd700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0ef8065000 0x7f0ef80674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:30.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.033+0000 7f0f06ffd700 1 -- 172.21.15.3:0/2069256704 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f0f10022070 con 0x7f0f1815d6e0 2023-03-03T06:38:30.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.037+0000 7f0f1659c700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0ef8065000 0x7f0ef80674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:30.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.037+0000 7f0f1659c700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0ef8065000 0x7f0ef80674b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0f08000f60 tx=0x7f0f08007e00 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:30.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.041+0000 7f0f06ffd700 1 -- 172.21.15.3:0/2069256704 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f0f10051c90 con 0x7f0f1815d6e0 2023-03-03T06:38:30.251 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.241+0000 7f0f1759e700 1 -- 172.21.15.3:0/2069256704 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f180c65f0 con 0x7f0ef8065000 2023-03-03T06:38:30.271 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.261+0000 7f0f06ffd700 1 -- 172.21.15.3:0/2069256704 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f0f180c65f0 con 0x7f0ef8065000 2023-03-03T06:38:30.272 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:38:30.272 INFO:teuthology.orchestra.run.smithi003.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:17", "first": "2023-03-03T06:28:01", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:30.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.265+0000 7f0f1759e700 1 -- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0ef8065000 msgr2=0x7f0ef80674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:30.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.265+0000 7f0f1759e700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0ef8065000 0x7f0ef80674b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0f08000f60 tx=0x7f0f08007e00 comp rx=0 tx=0).stop 2023-03-03T06:38:30.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.265+0000 7f0f1759e700 1 -- 172.21.15.3:0/2069256704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1815d6e0 msgr2=0x7f0f180ca6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:30.276 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.265+0000 7f0f1759e700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1815d6e0 0x7f0f180ca6c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f1806f190 tx=0x7f0f10004af0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.265+0000 7f0f1759e700 1 -- 172.21.15.3:0/2069256704 shutdown_connections 2023-03-03T06:38:30.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.265+0000 7f0f1759e700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f0ef8065000 0x7f0ef80674b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.265+0000 7f0f1759e700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f1815d6e0 0x7f0f180ca6c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.277 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.265+0000 7f0f1759e700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f0f18156ba0 0x7f0f180b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.265+0000 7f0f1759e700 1 --2- 172.21.15.3:0/2069256704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f180cfd80 0x7f0f1814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.265+0000 7f0f1759e700 1 -- 172.21.15.3:0/2069256704 >> 172.21.15.3:0/2069256704 conn(0x7f0f1806e5c0 msgr2=0x7f0f180b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:30.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.265+0000 7f0f1759e700 1 -- 172.21.15.3:0/2069256704 shutdown_connections 2023-03-03T06:38:30.278 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.265+0000 7f0f1759e700 1 -- 172.21.15.3:0/2069256704 wait complete. 2023-03-03T06:38:30.305 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:17", "first": "2023-03-03T06:28:01", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:30.306 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-03-03T00:00:00', 'created': '2023-03-03T06:27:17', 'first': '2023-03-03T06:28:01', 'last': '2023-03-03T06:38:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-03-03T06:38:30.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:30.307 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-03-03T06:38:30.339 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_28_00_UTC 2023-03-03T06:38:30.339 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_29_01_UTC 2023-03-03T06:38:30.339 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_30_00_UTC 2023-03-03T06:38:30.339 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_31_00_UTC 2023-03-03T06:38:30.340 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_32_00_UTC 2023-03-03T06:38:30.340 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_33_00_UTC 2023-03-03T06:38:30.340 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_34_00_UTC 2023-03-03T06:38:30.340 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_35_00_UTC 2023-03-03T06:38:30.340 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_36_00_UTC 2023-03-03T06:38:30.340 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_37_00_UTC 2023-03-03T06:38:30.340 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_38_00_UTC 2023-03-03T06:38:30.341 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-03-03-06_28_00_UTC', 'scheduled-2023-03-03-06_29_01_UTC', 'scheduled-2023-03-03-06_30_00_UTC', 'scheduled-2023-03-03-06_31_00_UTC', 'scheduled-2023-03-03-06_32_00_UTC', 'scheduled-2023-03-03-06_33_00_UTC', 'scheduled-2023-03-03-06_34_00_UTC', 'scheduled-2023-03-03-06_35_00_UTC', 'scheduled-2023-03-03-06_36_00_UTC', 'scheduled-2023-03-03-06_37_00_UTC', 'scheduled-2023-03-03-06_38_00_UTC'] 2023-03-03T06:38:30.342 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir4 2023-03-03T06:38:30.605 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.593+0000 7f530d6ae700 1 Processor -- start 2023-03-03T06:38:30.606 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.593+0000 7f530d6ae700 1 -- start start 2023-03-03T06:38:30.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.597+0000 7f530d6ae700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53080b6050 0x7f53080d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:30.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.597+0000 7f530d6ae700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f530815c960 0x7f53080b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:30.609 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.597+0000 7f530d6ae700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53080d4040 0x7f530806b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:30.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.597+0000 7f530d6ae700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53080d32c0 con 0x7f530815c960 2023-03-03T06:38:30.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.597+0000 7f530d6ae700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53080b6960 con 0x7f53080b6050 2023-03-03T06:38:30.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.597+0000 7f530d6ae700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53080d26b0 con 0x7f53080d4040 2023-03-03T06:38:30.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f53067fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f530815c960 0x7f53080b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:30.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f5306ffd700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53080b6050 0x7f53080d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:30.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f53067fc700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f530815c960 0x7f53080b6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38566/0 (socket says 172.21.15.3:38566) 2023-03-03T06:38:30.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f53067fc700 1 -- 172.21.15.3:0/3722119919 learned_addr learned my addr 172.21.15.3:0/3722119919 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:30.611 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f5307fff700 1 --2- 172.21.15.3:0/3722119919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53080d4040 0x7f530806b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:30.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f53067fc700 1 -- 172.21.15.3:0/3722119919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53080d4040 msgr2=0x7f530806b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:30.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f53067fc700 1 --2- 172.21.15.3:0/3722119919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53080d4040 0x7f530806b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f53067fc700 1 -- 172.21.15.3:0/3722119919 >> v2:172.21.15.32:3300/0 conn(0x7f53080b6050 msgr2=0x7f53080d3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:30.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f53067fc700 1 --2- 172.21.15.3:0/3722119919 >> v2:172.21.15.32:3300/0 conn(0x7f53080b6050 0x7f53080d3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.612 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f53067fc700 1 -- 172.21.15.3:0/3722119919 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53080d2990 con 0x7f530815c960 2023-03-03T06:38:30.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f53067fc700 1 --2- 172.21.15.3:0/3722119919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f530815c960 0x7f53080b6420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53080d1a10 tx=0x7f52fc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207521e9d236ee96 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:30.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f5305ffb700 1 -- 172.21.15.3:0/3722119919 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc011830 con 0x7f530815c960 2023-03-03T06:38:30.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f5305ffb700 1 -- 172.21.15.3:0/3722119919 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc011990 con 0x7f530815c960 2023-03-03T06:38:30.613 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f5305ffb700 1 -- 172.21.15.3:0/3722119919 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc011d20 con 0x7f530815c960 2023-03-03T06:38:30.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f530d6ae700 1 -- 172.21.15.3:0/3722119919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f530815c960 msgr2=0x7f53080b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:30.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.601+0000 7f530d6ae700 1 --2- 172.21.15.3:0/3722119919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f530815c960 0x7f53080b6420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53080d1a10 tx=0x7f52fc009bf0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.605+0000 7f530d6ae700 1 -- 172.21.15.3:0/3722119919 shutdown_connections 2023-03-03T06:38:30.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.605+0000 7f530d6ae700 1 --2- 172.21.15.3:0/3722119919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f53080d4040 0x7f530806b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.605+0000 7f530d6ae700 1 --2- 172.21.15.3:0/3722119919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f530815c960 0x7f53080b6420 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.605+0000 7f530d6ae700 1 --2- 172.21.15.3:0/3722119919 >> v2:172.21.15.32:3300/0 conn(0x7f53080b6050 0x7f53080d3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.605+0000 7f530d6ae700 1 -- 172.21.15.3:0/3722119919 >> 172.21.15.3:0/3722119919 conn(0x7f530806a9b0 msgr2=0x7f53080d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:30.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.605+0000 7f530d6ae700 1 -- 172.21.15.3:0/3722119919 shutdown_connections 2023-03-03T06:38:30.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.605+0000 7f530d6ae700 1 -- 172.21.15.3:0/3722119919 wait complete. 2023-03-03T06:38:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f530d6ae700 1 Processor -- start 2023-03-03T06:38:30.618 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f530d6ae700 1 -- start start 2023-03-03T06:38:30.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f530d6ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53080b6050 0x7f5308157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:30.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f530d6ae700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53080d4040 0x7f53080c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:30.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f530d6ae700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f530815c960 0x7f53081603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:30.620 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f530d6ae700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080b7fb0 con 0x7f53080d4040 2023-03-03T06:38:30.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f530d6ae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080ca750 con 0x7f53080b6050 2023-03-03T06:38:30.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f530d6ae700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53080caa30 con 0x7f530815c960 2023-03-03T06:38:30.621 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f53067fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53080d4040 0x7f53080c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f53067fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53080d4040 0x7f53080c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38572/0 (socket says 172.21.15.3:38572) 2023-03-03T06:38:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f53067fc700 1 -- 172.21.15.3:0/2814204632 learned_addr learned my addr 172.21.15.3:0/2814204632 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f5306ffd700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53080b6050 0x7f5308157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f5307fff700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f530815c960 0x7f53081603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f53067fc700 1 -- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f530815c960 msgr2=0x7f53081603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:30.622 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f53067fc700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f530815c960 0x7f53081603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f53067fc700 1 -- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53080b6050 msgr2=0x7f5308157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f53067fc700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53080b6050 0x7f5308157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.609+0000 7f53067fc700 1 -- 172.21.15.3:0/2814204632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc009570 con 0x7f53080d4040 2023-03-03T06:38:30.623 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.613+0000 7f53067fc700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53080d4040 0x7f53080c9c90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f52fc014040 tx=0x7f52fc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:30.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.613+0000 7f52ef7fe700 1 -- 172.21.15.3:0/2814204632 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc004520 con 0x7f53080d4040 2023-03-03T06:38:30.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.613+0000 7f52ef7fe700 1 -- 172.21.15.3:0/2814204632 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc01e070 con 0x7f53080d4040 2023-03-03T06:38:30.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.613+0000 7f530d6ae700 1 -- 172.21.15.3:0/2814204632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53080d2f50 con 0x7f53080d4040 2023-03-03T06:38:30.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.613+0000 7f52ef7fe700 1 -- 172.21.15.3:0/2814204632 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc003900 con 0x7f53080d4040 2023-03-03T06:38:30.624 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.613+0000 7f530d6ae700 1 -- 172.21.15.3:0/2814204632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53080d2c70 con 0x7f53080d4040 2023-03-03T06:38:30.625 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.613+0000 7f530d6ae700 1 -- 172.21.15.3:0/2814204632 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53080cf7f0 con 0x7f53080d4040 2023-03-03T06:38:30.627 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.617+0000 7f52ef7fe700 1 -- 172.21.15.3:0/2814204632 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f52fc00f050 con 0x7f53080d4040 2023-03-03T06:38:30.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.617+0000 7f52ef7fe700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f52f4065000 0x7f52f40674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:30.629 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.617+0000 7f52ef7fe700 1 -- 172.21.15.3:0/2814204632 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f52fc020070 con 0x7f53080d4040 2023-03-03T06:38:30.630 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.621+0000 7f5306ffd700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f52f4065000 0x7f52f40674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:30.632 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.621+0000 7f5306ffd700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f52f4065000 0x7f52f40674b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f53080cf4a0 tx=0x7f52f8008040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:30.634 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.625+0000 7f52ef7fe700 1 -- 172.21.15.3:0/2814204632 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f52fc0513b0 con 0x7f53080d4040 2023-03-03T06:38:30.833 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.821+0000 7f530d6ae700 1 -- 172.21.15.3:0/2814204632 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53080cabf0 con 0x7f52f4065000 2023-03-03T06:38:30.854 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.845+0000 7f52ef7fe700 1 -- 172.21.15.3:0/2814204632 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f53080cabf0 con 0x7f52f4065000 2023-03-03T06:38:30.855 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:38:30.855 INFO:teuthology.orchestra.run.smithi003.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:17", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:30.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.849+0000 7f530d6ae700 1 -- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f52f4065000 msgr2=0x7f52f40674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:30.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.849+0000 7f530d6ae700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f52f4065000 0x7f52f40674b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f53080cf4a0 tx=0x7f52f8008040 comp rx=0 tx=0).stop 2023-03-03T06:38:30.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.849+0000 7f530d6ae700 1 -- 172.21.15.3:0/2814204632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53080d4040 msgr2=0x7f53080c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:30.859 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.849+0000 7f530d6ae700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53080d4040 0x7f53080c9c90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f52fc014040 tx=0x7f52fc004770 comp rx=0 tx=0).stop 2023-03-03T06:38:30.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.849+0000 7f530d6ae700 1 -- 172.21.15.3:0/2814204632 shutdown_connections 2023-03-03T06:38:30.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.849+0000 7f530d6ae700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f52f4065000 0x7f52f40674b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.860 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.849+0000 7f530d6ae700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f530815c960 0x7f53081603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.849+0000 7f530d6ae700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53080d4040 0x7f53080c9c90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.849+0000 7f530d6ae700 1 --2- 172.21.15.3:0/2814204632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53080b6050 0x7f5308157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:30.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.849+0000 7f530d6ae700 1 -- 172.21.15.3:0/2814204632 >> 172.21.15.3:0/2814204632 conn(0x7f530806a9b0 msgr2=0x7f530815bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:30.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.849+0000 7f530d6ae700 1 -- 172.21.15.3:0/2814204632 shutdown_connections 2023-03-03T06:38:30.861 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:30.849+0000 7f530d6ae700 1 -- 172.21.15.3:0/2814204632 wait complete. 2023-03-03T06:38:30.892 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:17", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:30.892 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-03-03T00:00:00', 'created': '2023-03-03T06:27:17', 'first': '2023-03-03T06:28:00', 'last': '2023-03-03T06:38:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-03-03T06:38:30.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:30.893 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-03-03T06:38:30.920 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_28_00_UTC 2023-03-03T06:38:30.921 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_29_00_UTC 2023-03-03T06:38:30.921 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_30_01_UTC 2023-03-03T06:38:30.921 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_31_00_UTC 2023-03-03T06:38:30.921 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_32_00_UTC 2023-03-03T06:38:30.921 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_33_00_UTC 2023-03-03T06:38:30.921 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_34_01_UTC 2023-03-03T06:38:30.922 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_35_01_UTC 2023-03-03T06:38:30.922 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_36_00_UTC 2023-03-03T06:38:30.922 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_37_00_UTC 2023-03-03T06:38:30.922 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_38_01_UTC 2023-03-03T06:38:30.923 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-03-03-06_28_00_UTC', 'scheduled-2023-03-03-06_29_00_UTC', 'scheduled-2023-03-03-06_30_01_UTC', 'scheduled-2023-03-03-06_31_00_UTC', 'scheduled-2023-03-03-06_32_00_UTC', 'scheduled-2023-03-03-06_33_00_UTC', 'scheduled-2023-03-03-06_34_01_UTC', 'scheduled-2023-03-03-06_35_01_UTC', 'scheduled-2023-03-03-06_36_00_UTC', 'scheduled-2023-03-03-06_37_00_UTC', 'scheduled-2023-03-03-06_38_01_UTC'] 2023-03-03T06:38:30.923 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir5 2023-03-03T06:38:31.187 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.177+0000 7fd27cc2d700 1 Processor -- start 2023-03-03T06:38:31.188 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.177+0000 7fd27cc2d700 1 -- start start 2023-03-03T06:38:31.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd27cc2d700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd278156ba0 0x7fd278071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:31.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd27cc2d700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd27815d6e0 0x7fd2780cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:31.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd27cc2d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd2780cfd80 0x7fd2780d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:31.192 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd27cc2d700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2780cf3b0 con 0x7fd27815d6e0 2023-03-03T06:38:31.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd27cc2d700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd278156f70 con 0x7fd278156ba0 2023-03-03T06:38:31.193 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd27cc2d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2780cd320 con 0x7fd2780cfd80 2023-03-03T06:38:31.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd276ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd27815d6e0 0x7fd2780cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:31.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd276ffd700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd27815d6e0 0x7fd2780cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38582/0 (socket says 172.21.15.3:38582) 2023-03-03T06:38:31.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd276ffd700 1 -- 172.21.15.3:0/3246331568 learned_addr learned my addr 172.21.15.3:0/3246331568 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:31.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd2777fe700 1 --2- 172.21.15.3:0/3246331568 >> v2:172.21.15.32:3300/0 conn(0x7fd278156ba0 0x7fd278071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:31.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd277fff700 1 --2- 172.21.15.3:0/3246331568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd2780cfd80 0x7fd2780d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:31.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd276ffd700 1 -- 172.21.15.3:0/3246331568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd2780cfd80 msgr2=0x7fd2780d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:31.194 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd276ffd700 1 --2- 172.21.15.3:0/3246331568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd2780cfd80 0x7fd2780d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd276ffd700 1 -- 172.21.15.3:0/3246331568 >> v2:172.21.15.32:3300/0 conn(0x7fd278156ba0 msgr2=0x7fd278071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:31.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd276ffd700 1 --2- 172.21.15.3:0/3246331568 >> v2:172.21.15.32:3300/0 conn(0x7fd278156ba0 0x7fd278071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd276ffd700 1 -- 172.21.15.3:0/3246331568 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2780cd600 con 0x7fd27815d6e0 2023-03-03T06:38:31.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd277fff700 1 --2- 172.21.15.3:0/3246331568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd2780cfd80 0x7fd2780d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:38:31.195 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.181+0000 7fd2777fe700 1 --2- 172.21.15.3:0/3246331568 >> v2:172.21.15.32:3300/0 conn(0x7fd278156ba0 0x7fd278071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:38:31.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.185+0000 7fd276ffd700 1 --2- 172.21.15.3:0/3246331568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd27815d6e0 0x7fd2780cf840 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd278072100 tx=0x7fd26c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bb3d576286ee8b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:31.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.185+0000 7fd2767fc700 1 -- 172.21.15.3:0/3246331568 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00b8a0 con 0x7fd27815d6e0 2023-03-03T06:38:31.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.185+0000 7fd2767fc700 1 -- 172.21.15.3:0/3246331568 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26c00ba00 con 0x7fd27815d6e0 2023-03-03T06:38:31.196 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.185+0000 7fd2767fc700 1 -- 172.21.15.3:0/3246331568 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00bd90 con 0x7fd27815d6e0 2023-03-03T06:38:31.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.185+0000 7fd27cc2d700 1 -- 172.21.15.3:0/3246331568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd27815d6e0 msgr2=0x7fd2780cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:31.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.185+0000 7fd27cc2d700 1 --2- 172.21.15.3:0/3246331568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd27815d6e0 0x7fd2780cf840 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd278072100 tx=0x7fd26c009bf0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.185+0000 7fd27cc2d700 1 -- 172.21.15.3:0/3246331568 shutdown_connections 2023-03-03T06:38:31.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.185+0000 7fd27cc2d700 1 --2- 172.21.15.3:0/3246331568 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd2780cfd80 0x7fd2780d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.185+0000 7fd27cc2d700 1 --2- 172.21.15.3:0/3246331568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd27815d6e0 0x7fd2780cf840 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.185+0000 7fd27cc2d700 1 --2- 172.21.15.3:0/3246331568 >> v2:172.21.15.32:3300/0 conn(0x7fd278156ba0 0x7fd278071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.197 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.185+0000 7fd27cc2d700 1 -- 172.21.15.3:0/3246331568 >> 172.21.15.3:0/3246331568 conn(0x7fd27806e5c0 msgr2=0x7fd27806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:31.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.185+0000 7fd27cc2d700 1 -- 172.21.15.3:0/3246331568 shutdown_connections 2023-03-03T06:38:31.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.185+0000 7fd27cc2d700 1 -- 172.21.15.3:0/3246331568 wait complete. 2023-03-03T06:38:31.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd27cc2d700 1 Processor -- start 2023-03-03T06:38:31.198 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd27cc2d700 1 -- start start 2023-03-03T06:38:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd27cc2d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2780cfd80 0x7fd2780b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd27cc2d700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd278156ba0 0x7fd27815dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd27cc2d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27815d6e0 0x7fd27815e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd27cc2d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27815f0a0 con 0x7fd2780cfd80 2023-03-03T06:38:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd27cc2d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2780b6440 con 0x7fd27815d6e0 2023-03-03T06:38:31.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd27cc2d700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27814e9d0 con 0x7fd278156ba0 2023-03-03T06:38:31.201 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd2777fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2780cfd80 0x7fd2780b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:31.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd2777fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2780cfd80 0x7fd2780b2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38584/0 (socket says 172.21.15.3:38584) 2023-03-03T06:38:31.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd2777fe700 1 -- 172.21.15.3:0/2793580449 learned_addr learned my addr 172.21.15.3:0/2793580449 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:31.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd277fff700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27815d6e0 0x7fd27815e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:31.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd276ffd700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd278156ba0 0x7fd27815dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:31.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd2777fe700 1 -- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd278156ba0 msgr2=0x7fd27815dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:31.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd2777fe700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd278156ba0 0x7fd27815dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.202 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd2777fe700 1 -- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27815d6e0 msgr2=0x7fd27815e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:31.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd2777fe700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27815d6e0 0x7fd27815e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.189+0000 7fd2777fe700 1 -- 172.21.15.3:0/2793580449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c009570 con 0x7fd2780cfd80 2023-03-03T06:38:31.203 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.193+0000 7fd2777fe700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2780cfd80 0x7fd2780b2910 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd26800e810 tx=0x7fd26800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:31.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.193+0000 7fd25ffff700 1 -- 172.21.15.3:0/2793580449 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd268004230 con 0x7fd2780cfd80 2023-03-03T06:38:31.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.193+0000 7fd25ffff700 1 -- 172.21.15.3:0/2793580449 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd268004390 con 0x7fd2780cfd80 2023-03-03T06:38:31.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.193+0000 7fd25ffff700 1 -- 172.21.15.3:0/2793580449 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2680036c0 con 0x7fd2780cfd80 2023-03-03T06:38:31.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.193+0000 7fd27cc2d700 1 -- 172.21.15.3:0/2793580449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2780717c0 con 0x7fd2780cfd80 2023-03-03T06:38:31.204 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.193+0000 7fd27cc2d700 1 -- 172.21.15.3:0/2793580449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2780cd8e0 con 0x7fd2780cfd80 2023-03-03T06:38:31.205 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.193+0000 7fd27cc2d700 1 -- 172.21.15.3:0/2793580449 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2780cef40 con 0x7fd2780cfd80 2023-03-03T06:38:31.206 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.197+0000 7fd25ffff700 1 -- 172.21.15.3:0/2793580449 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7fd268022070 con 0x7fd2780cfd80 2023-03-03T06:38:31.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.197+0000 7fd25ffff700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd264065000 0x7fd2640674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:31.209 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.197+0000 7fd25ffff700 1 -- 172.21.15.3:0/2793580449 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fd268081dd0 con 0x7fd2780cfd80 2023-03-03T06:38:31.210 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.201+0000 7fd276ffd700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd264065000 0x7fd2640674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:31.211 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.201+0000 7fd276ffd700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd264065000 0x7fd2640674b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd26c000c00 tx=0x7fd26c008030 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:31.213 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.201+0000 7fd25ffff700 1 -- 172.21.15.3:0/2793580449 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd26804fad0 con 0x7fd2780cfd80 2023-03-03T06:38:31.410 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.397+0000 7fd27cc2d700 1 -- 172.21.15.3:0/2793580449 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd278073e90 con 0x7fd264065000 2023-03-03T06:38:31.431 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.421+0000 7fd25ffff700 1 -- 172.21.15.3:0/2793580449 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fd278073e90 con 0x7fd264065000 2023-03-03T06:38:31.432 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:38:31.432 INFO:teuthology.orchestra.run.smithi003.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:18", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:31.435 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.425+0000 7fd27cc2d700 1 -- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd264065000 msgr2=0x7fd2640674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:31.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.425+0000 7fd27cc2d700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd264065000 0x7fd2640674b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd26c000c00 tx=0x7fd26c008030 comp rx=0 tx=0).stop 2023-03-03T06:38:31.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.425+0000 7fd27cc2d700 1 -- 172.21.15.3:0/2793580449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2780cfd80 msgr2=0x7fd2780b2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:31.436 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.425+0000 7fd27cc2d700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2780cfd80 0x7fd2780b2910 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd26800e810 tx=0x7fd26800ebd0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.425+0000 7fd27cc2d700 1 -- 172.21.15.3:0/2793580449 shutdown_connections 2023-03-03T06:38:31.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.425+0000 7fd27cc2d700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd264065000 0x7fd2640674b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.425+0000 7fd27cc2d700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd27815d6e0 0x7fd27815e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.425+0000 7fd27cc2d700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd278156ba0 0x7fd27815dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.425+0000 7fd27cc2d700 1 --2- 172.21.15.3:0/2793580449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2780cfd80 0x7fd2780b2910 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.437 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.425+0000 7fd27cc2d700 1 -- 172.21.15.3:0/2793580449 >> 172.21.15.3:0/2793580449 conn(0x7fd27806e5c0 msgr2=0x7fd278151040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:31.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.425+0000 7fd27cc2d700 1 -- 172.21.15.3:0/2793580449 shutdown_connections 2023-03-03T06:38:31.438 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.425+0000 7fd27cc2d700 1 -- 172.21.15.3:0/2793580449 wait complete. 2023-03-03T06:38:31.471 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:18", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:31.471 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-03-03T00:00:00', 'created': '2023-03-03T06:27:18', 'first': '2023-03-03T06:28:00', 'last': '2023-03-03T06:38:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-03-03T06:38:31.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:31.472 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-03-03T06:38:31.499 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_28_00_UTC 2023-03-03T06:38:31.499 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_29_00_UTC 2023-03-03T06:38:31.499 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_30_00_UTC 2023-03-03T06:38:31.499 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_31_00_UTC 2023-03-03T06:38:31.500 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_32_01_UTC 2023-03-03T06:38:31.500 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_33_00_UTC 2023-03-03T06:38:31.500 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_34_00_UTC 2023-03-03T06:38:31.500 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_35_01_UTC 2023-03-03T06:38:31.500 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_36_01_UTC 2023-03-03T06:38:31.500 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_37_00_UTC 2023-03-03T06:38:31.500 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_38_00_UTC 2023-03-03T06:38:31.501 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-03-03-06_28_00_UTC', 'scheduled-2023-03-03-06_29_00_UTC', 'scheduled-2023-03-03-06_30_00_UTC', 'scheduled-2023-03-03-06_31_00_UTC', 'scheduled-2023-03-03-06_32_01_UTC', 'scheduled-2023-03-03-06_33_00_UTC', 'scheduled-2023-03-03-06_34_00_UTC', 'scheduled-2023-03-03-06_35_01_UTC', 'scheduled-2023-03-03-06_36_01_UTC', 'scheduled-2023-03-03-06_37_00_UTC', 'scheduled-2023-03-03-06_38_00_UTC'] 2023-03-03T06:38:31.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir6 2023-03-03T06:38:31.767 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.757+0000 7f67b9543700 1 Processor -- start 2023-03-03T06:38:31.768 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.757+0000 7f67b9543700 1 -- start start 2023-03-03T06:38:31.771 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.761+0000 7f67b9543700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67b4156ba0 0x7f67b4071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:31.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.761+0000 7f67b9543700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67b40d0b30 0x7f67b40d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:31.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.761+0000 7f67b9543700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b40cf3b0 con 0x7f67b415d6e0 2023-03-03T06:38:31.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.761+0000 7f67b9543700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b40d14f0 con 0x7f67b40d0b30 2023-03-03T06:38:31.772 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.761+0000 7f67b9543700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b4156f70 con 0x7f67b4156ba0 2023-03-03T06:38:31.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.761+0000 7f67b37fe700 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f67b415d6e0 0x7f67b40cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.3:34034/0 (socket says 172.21.15.3:34034) 2023-03-03T06:38:31.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.761+0000 7f67b37fe700 1 -- 172.21.15.3:0/3218868896 learned_addr learned my addr 172.21.15.3:0/3218868896 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:38:31.773 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.761+0000 7f67b8d42700 1 --2- 172.21.15.3:0/3218868896 >> v2:172.21.15.32:3300/0 conn(0x7f67b40d0b30 0x7f67b40d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:31.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.761+0000 7f67b3fff700 1 --2- 172.21.15.3:0/3218868896 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67b4156ba0 0x7f67b4071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:31.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.761+0000 7f67b2ffd700 1 -- 172.21.15.3:0/3218868896 <== mon.1 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489679116 0 0) 0x7f67b40cf3b0 con 0x7f67b415d6e0 2023-03-03T06:38:31.774 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.761+0000 7f67b2ffd700 1 -- 172.21.15.3:0/3218868896 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f679c0035d0 con 0x7f67b415d6e0 2023-03-03T06:38:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.765+0000 7f67b8d42700 1 -- 172.21.15.3:0/3218868896 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67b4156ba0 msgr2=0x7f67b4071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.765+0000 7f67b8d42700 1 --2- 172.21.15.3:0/3218868896 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67b4156ba0 0x7f67b4071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.765+0000 7f67b2ffd700 1 -- 172.21.15.3:0/3218868896 <== mon.1 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1668749993 0 0) 0x7f679c0035d0 con 0x7f67b415d6e0 2023-03-03T06:38:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.765+0000 7f67b8d42700 1 -- 172.21.15.3:0/3218868896 >> v1:172.21.15.32:6789/0 conn(0x7f67b415d6e0 legacy=0x7f67b40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:31.775 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.765+0000 7f67b8d42700 1 -- 172.21.15.3:0/3218868896 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b40cd320 con 0x7f67b40d0b30 2023-03-03T06:38:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.765+0000 7f67b8d42700 1 --2- 172.21.15.3:0/3218868896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b40d0b30 0x7f67b40d0fb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009850 tx=0x7f67ac009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=811a4eb447e32c00 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.765+0000 7f67b2ffd700 1 -- 172.21.15.3:0/3218868896 <== mon.1 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a8001670 con 0x7f67b40d0b30 2023-03-03T06:38:31.776 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.765+0000 7f67b2ffd700 1 -- 172.21.15.3:0/3218868896 <== mon.1 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac004030 con 0x7f67b40d0b30 2023-03-03T06:38:31.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.765+0000 7f67b2ffd700 1 -- 172.21.15.3:0/3218868896 <== mon.1 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac0043e0 con 0x7f67b40d0b30 2023-03-03T06:38:31.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.765+0000 7f67b9543700 1 -- 172.21.15.3:0/3218868896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b40d0b30 msgr2=0x7f67b40d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:31.777 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.765+0000 7f67b9543700 1 --2- 172.21.15.3:0/3218868896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b40d0b30 0x7f67b40d0fb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009850 tx=0x7f67ac009b60 comp rx=0 tx=0).stop 2023-03-03T06:38:31.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.769+0000 7f67b9543700 1 -- 172.21.15.3:0/3218868896 shutdown_connections 2023-03-03T06:38:31.778 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.769+0000 7f67b9543700 1 --2- 172.21.15.3:0/3218868896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b40d0b30 0x7f67b40d0fb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.769+0000 7f67b9543700 1 --2- 172.21.15.3:0/3218868896 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67b4156ba0 0x7f67b4071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.769+0000 7f67b9543700 1 -- 172.21.15.3:0/3218868896 >> 172.21.15.3:0/3218868896 conn(0x7f67b406e5c0 msgr2=0x7f67b406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:31.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.769+0000 7f67b9543700 1 -- 172.21.15.3:0/3218868896 shutdown_connections 2023-03-03T06:38:31.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.769+0000 7f67b9543700 1 -- 172.21.15.3:0/3218868896 wait complete. 2023-03-03T06:38:31.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.773+0000 7f67b9543700 1 Processor -- start 2023-03-03T06:38:31.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.773+0000 7f67b9543700 1 -- start start 2023-03-03T06:38:31.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.773+0000 7f67b9543700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b40d0b30 0x7f67b40cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:31.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.773+0000 7f67b9543700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67b4156ba0 0x7f67b4160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:31.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.773+0000 7f67b9543700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b415d6e0 0x7f67b4160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:31.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.773+0000 7f67b9543700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b40b61c0 con 0x7f67b40d0b30 2023-03-03T06:38:31.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.773+0000 7f67b9543700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b40cca10 con 0x7f67b415d6e0 2023-03-03T06:38:31.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.773+0000 7f67b9543700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b40b4fb0 con 0x7f67b4156ba0 2023-03-03T06:38:31.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.773+0000 7f67b3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b40d0b30 0x7f67b40cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:31.786 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.773+0000 7f67b3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b40d0b30 0x7f67b40cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38586/0 (socket says 172.21.15.3:38586) 2023-03-03T06:38:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.773+0000 7f67b3fff700 1 -- 172.21.15.3:0/426032919 learned_addr learned my addr 172.21.15.3:0/426032919 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.773+0000 7f67b8d42700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b415d6e0 0x7f67b4160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.773+0000 7f67b37fe700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67b4156ba0 0x7f67b4160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.777+0000 7f67b3fff700 1 -- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67b4156ba0 msgr2=0x7f67b4160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.777+0000 7f67b3fff700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67b4156ba0 0x7f67b4160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.777+0000 7f67b3fff700 1 -- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b415d6e0 msgr2=0x7f67b4160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:31.787 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.777+0000 7f67b3fff700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b415d6e0 0x7f67b4160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:31.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.777+0000 7f67b3fff700 1 -- 172.21.15.3:0/426032919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac0094b0 con 0x7f67b40d0b30 2023-03-03T06:38:31.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.777+0000 7f67b37fe700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67b4156ba0 0x7f67b4160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:38:31.788 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.777+0000 7f67b3fff700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b40d0b30 0x7f67b40cc4d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f67a400efc0 tx=0x7f67a400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:31.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.777+0000 7f67b0ff9700 1 -- 172.21.15.3:0/426032919 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a400cc00 con 0x7f67b40d0b30 2023-03-03T06:38:31.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.777+0000 7f67b0ff9700 1 -- 172.21.15.3:0/426032919 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a400f040 con 0x7f67b40d0b30 2023-03-03T06:38:31.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.777+0000 7f67b0ff9700 1 -- 172.21.15.3:0/426032919 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4014480 con 0x7f67b40d0b30 2023-03-03T06:38:31.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.777+0000 7f67b9543700 1 -- 172.21.15.3:0/426032919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b40cd8e0 con 0x7f67b40d0b30 2023-03-03T06:38:31.789 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.777+0000 7f67b9543700 1 -- 172.21.15.3:0/426032919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b40cd600 con 0x7f67b40d0b30 2023-03-03T06:38:31.790 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.781+0000 7f67b9543700 1 -- 172.21.15.3:0/426032919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b40717c0 con 0x7f67b40d0b30 2023-03-03T06:38:31.793 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.781+0000 7f67b0ff9700 1 -- 172.21.15.3:0/426032919 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f67a40145e0 con 0x7f67b40d0b30 2023-03-03T06:38:31.795 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.785+0000 7f67b0ff9700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f67a0065000 0x7f67a00674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:31.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.785+0000 7f67b0ff9700 1 -- 172.21.15.3:0/426032919 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f67a4082f70 con 0x7f67b40d0b30 2023-03-03T06:38:31.796 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.785+0000 7f67b37fe700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f67a0065000 0x7f67a00674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:31.797 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.785+0000 7f67b37fe700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f67a0065000 0x7f67a00674b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67a8001860 tx=0x7f67a80015f0 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:31.801 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.789+0000 7f67b0ff9700 1 -- 172.21.15.3:0/426032919 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f67a4050cd0 con 0x7f67b40d0b30 2023-03-03T06:38:32.001 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:31.989+0000 7f67b9543700 1 -- 172.21.15.3:0/426032919 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67b40b6430 con 0x7f67a0065000 2023-03-03T06:38:32.021 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.009+0000 7f67b0ff9700 1 -- 172.21.15.3:0/426032919 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f67b40b6430 con 0x7f67a0065000 2023-03-03T06:38:32.022 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:38:32.022 INFO:teuthology.orchestra.run.smithi003.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:19", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:32.025 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.013+0000 7f67b9543700 1 -- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f67a0065000 msgr2=0x7f67a00674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:32.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.013+0000 7f67b9543700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f67a0065000 0x7f67a00674b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67a8001860 tx=0x7f67a80015f0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.013+0000 7f67b9543700 1 -- 172.21.15.3:0/426032919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b40d0b30 msgr2=0x7f67b40cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:32.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.013+0000 7f67b9543700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b40d0b30 0x7f67b40cc4d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f67a400efc0 tx=0x7f67a400c0c0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.026 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.013+0000 7f67b9543700 1 -- 172.21.15.3:0/426032919 shutdown_connections 2023-03-03T06:38:32.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.013+0000 7f67b9543700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f67a0065000 0x7f67a00674b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.013+0000 7f67b9543700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67b415d6e0 0x7f67b4160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.013+0000 7f67b9543700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f67b4156ba0 0x7f67b4160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.013+0000 7f67b9543700 1 --2- 172.21.15.3:0/426032919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b40d0b30 0x7f67b40cc4d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.027 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.013+0000 7f67b9543700 1 -- 172.21.15.3:0/426032919 >> 172.21.15.3:0/426032919 conn(0x7f67b406e5c0 msgr2=0x7f67b40cf740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:32.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.017+0000 7f67b9543700 1 -- 172.21.15.3:0/426032919 shutdown_connections 2023-03-03T06:38:32.028 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.017+0000 7f67b9543700 1 -- 172.21.15.3:0/426032919 wait complete. 2023-03-03T06:38:32.065 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:19", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:32.065 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-03-03T00:00:00', 'created': '2023-03-03T06:27:19', 'first': '2023-03-03T06:28:00', 'last': '2023-03-03T06:38:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-03-03T06:38:32.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:32.066 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-03-03T06:38:32.097 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_28_00_UTC 2023-03-03T06:38:32.097 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_29_00_UTC 2023-03-03T06:38:32.097 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_30_00_UTC 2023-03-03T06:38:32.098 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_31_01_UTC 2023-03-03T06:38:32.098 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_32_00_UTC 2023-03-03T06:38:32.098 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_33_00_UTC 2023-03-03T06:38:32.098 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_34_01_UTC 2023-03-03T06:38:32.098 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_35_00_UTC 2023-03-03T06:38:32.098 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_36_00_UTC 2023-03-03T06:38:32.098 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_37_00_UTC 2023-03-03T06:38:32.098 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_38_00_UTC 2023-03-03T06:38:32.099 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-03-03-06_28_00_UTC', 'scheduled-2023-03-03-06_29_00_UTC', 'scheduled-2023-03-03-06_30_00_UTC', 'scheduled-2023-03-03-06_31_01_UTC', 'scheduled-2023-03-03-06_32_00_UTC', 'scheduled-2023-03-03-06_33_00_UTC', 'scheduled-2023-03-03-06_34_01_UTC', 'scheduled-2023-03-03-06_35_00_UTC', 'scheduled-2023-03-03-06_36_00_UTC', 'scheduled-2023-03-03-06_37_00_UTC', 'scheduled-2023-03-03-06_38_00_UTC'] 2023-03-03T06:38:32.100 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir7 2023-03-03T06:38:32.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.353+0000 7fd1fd4ee700 1 Processor -- start 2023-03-03T06:38:32.364 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.353+0000 7fd1fd4ee700 1 -- start start 2023-03-03T06:38:32.366 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.357+0000 7fd1fd4ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1f806db10 0x7fd1f80b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:32.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.357+0000 7fd1fd4ee700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f80cedd0 con 0x7fd1f80cfd10 2023-03-03T06:38:32.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.357+0000 7fd1fd4ee700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f815ce40 con 0x7fd1f80cb190 2023-03-03T06:38:32.367 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.357+0000 7fd1fd4ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f815cb60 con 0x7fd1f806db10 2023-03-03T06:38:32.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.357+0000 7fd1f67fc700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd1f80cfd10 0x7fd1f806e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51068/0 (socket says 172.21.15.3:51068) 2023-03-03T06:38:32.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.357+0000 7fd1f67fc700 1 -- 172.21.15.3:0/3332485824 learned_addr learned my addr 172.21.15.3:0/3332485824 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:38:32.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.357+0000 7fd1f6ffd700 1 --2- 172.21.15.3:0/3332485824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1f806db10 0x7fd1f80b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:32.368 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.357+0000 7fd1f5ffb700 1 -- 172.21.15.3:0/3332485824 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039505408 0 0) 0x7fd1f80cedd0 con 0x7fd1f80cfd10 2023-03-03T06:38:32.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.357+0000 7fd1f6ffd700 1 -- 172.21.15.3:0/3332485824 >> v1:172.21.15.32:6789/0 conn(0x7fd1f80cb190 legacy=0x7fd1f8163340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:32.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.357+0000 7fd1f6ffd700 1 -- 172.21.15.3:0/3332485824 >> v1:172.21.15.3:6789/0 conn(0x7fd1f80cfd10 legacy=0x7fd1f806e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:32.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.357+0000 7fd1f6ffd700 1 -- 172.21.15.3:0/3332485824 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f80ce780 con 0x7fd1f806db10 2023-03-03T06:38:32.369 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.357+0000 7fd1f6ffd700 1 --2- 172.21.15.3:0/3332485824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1f806db10 0x7fd1f80b1c40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009870 tx=0x7fd1e8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e0d62b84963b5c8 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:32.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.361+0000 7fd1f5ffb700 1 -- 172.21.15.3:0/3332485824 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8005e50 con 0x7fd1f806db10 2023-03-03T06:38:32.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.361+0000 7fd1f5ffb700 1 -- 172.21.15.3:0/3332485824 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e8006030 con 0x7fd1f806db10 2023-03-03T06:38:32.370 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.361+0000 7fd1f5ffb700 1 -- 172.21.15.3:0/3332485824 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80063c0 con 0x7fd1f806db10 2023-03-03T06:38:32.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.361+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3332485824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1f806db10 msgr2=0x7fd1f80b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:32.371 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.361+0000 7fd1fd4ee700 1 --2- 172.21.15.3:0/3332485824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1f806db10 0x7fd1f80b1c40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009870 tx=0x7fd1e8009b80 comp rx=0 tx=0).stop 2023-03-03T06:38:32.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.361+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3332485824 shutdown_connections 2023-03-03T06:38:32.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.361+0000 7fd1fd4ee700 1 --2- 172.21.15.3:0/3332485824 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1f806db10 0x7fd1f80b1c40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.361+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3332485824 >> 172.21.15.3:0/3332485824 conn(0x7fd1f80ccc40 msgr2=0x7fd1f80cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:32.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.361+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3332485824 shutdown_connections 2023-03-03T06:38:32.372 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.361+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3332485824 wait complete. 2023-03-03T06:38:32.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.361+0000 7fd1fd4ee700 1 Processor -- start 2023-03-03T06:38:32.374 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1fd4ee700 1 -- start start 2023-03-03T06:38:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1fd4ee700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1f806db10 0x7fd1f8155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1fd4ee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1f80cb190 0x7fd1f815adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1fd4ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f80cfd10 0x7fd1f815b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1fd4ee700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f8156010 con 0x7fd1f80cb190 2023-03-03T06:38:32.375 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1fd4ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f80c4270 con 0x7fd1f80cfd10 2023-03-03T06:38:32.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1fd4ee700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f80c4550 con 0x7fd1f806db10 2023-03-03T06:38:32.376 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1f67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1f80cb190 0x7fd1f815adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:32.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1f67fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1f80cb190 0x7fd1f815adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38598/0 (socket says 172.21.15.3:38598) 2023-03-03T06:38:32.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1f67fc700 1 -- 172.21.15.3:0/3573875882 learned_addr learned my addr 172.21.15.3:0/3573875882 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:32.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1f6ffd700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1f806db10 0x7fd1f8155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:32.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1f7fff700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f80cfd10 0x7fd1f815b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:32.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1f67fc700 1 -- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1f806db10 msgr2=0x7fd1f8155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:32.377 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1f67fc700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1f806db10 0x7fd1f8155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1f67fc700 1 -- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f80cfd10 msgr2=0x7fd1f815b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:32.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1f67fc700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f80cfd10 0x7fd1f815b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1f67fc700 1 -- 172.21.15.3:0/3573875882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e8009450 con 0x7fd1f80cb190 2023-03-03T06:38:32.378 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1f67fc700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1f80cb190 0x7fd1f815adb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c4e0 tx=0x7fd1ec00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:32.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1df7fe700 1 -- 172.21.15.3:0/3573875882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec004ca0 con 0x7fd1f80cb190 2023-03-03T06:38:32.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.365+0000 7fd1df7fe700 1 -- 172.21.15.3:0/3573875882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec004e20 con 0x7fd1f80cb190 2023-03-03T06:38:32.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.369+0000 7fd1df7fe700 1 -- 172.21.15.3:0/3573875882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec007310 con 0x7fd1f80cb190 2023-03-03T06:38:32.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.369+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3573875882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f80cace0 con 0x7fd1f80cb190 2023-03-03T06:38:32.379 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.369+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3573875882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f80cea60 con 0x7fd1f80cb190 2023-03-03T06:38:32.380 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.369+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3573875882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f8158540 con 0x7fd1f80cb190 2023-03-03T06:38:32.382 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.373+0000 7fd1df7fe700 1 -- 172.21.15.3:0/3573875882 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7fd1ec01a020 con 0x7fd1f80cb190 2023-03-03T06:38:32.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.373+0000 7fd1df7fe700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd1e4065000 0x7fd1e40674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:32.384 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.373+0000 7fd1df7fe700 1 -- 172.21.15.3:0/3573875882 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7fd1ec081ea0 con 0x7fd1f80cb190 2023-03-03T06:38:32.385 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.373+0000 7fd1f6ffd700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd1e4065000 0x7fd1e40674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.377+0000 7fd1f6ffd700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd1e4065000 0x7fd1e40674b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8000c00 tx=0x7fd1e8008070 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:32.390 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.381+0000 7fd1df7fe700 1 -- 172.21.15.3:0/3573875882 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd1ec04fcb0 con 0x7fd1f80cb190 2023-03-03T06:38:32.589 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.577+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3573875882 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1f804e180 con 0x7fd1e4065000 2023-03-03T06:38:32.610 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.601+0000 7fd1df7fe700 1 -- 172.21.15.3:0/3573875882 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fd1f804e180 con 0x7fd1e4065000 2023-03-03T06:38:32.611 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:38:32.611 INFO:teuthology.orchestra.run.smithi003.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:19", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:32.614 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.605+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd1e4065000 msgr2=0x7fd1e40674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:32.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.605+0000 7fd1fd4ee700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd1e4065000 0x7fd1e40674b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8000c00 tx=0x7fd1e8008070 comp rx=0 tx=0).stop 2023-03-03T06:38:32.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.605+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3573875882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1f80cb190 msgr2=0x7fd1f815adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:32.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.605+0000 7fd1fd4ee700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1f80cb190 0x7fd1f815adb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c4e0 tx=0x7fd1ec00c7f0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.605+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3573875882 shutdown_connections 2023-03-03T06:38:32.615 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.605+0000 7fd1fd4ee700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7fd1e4065000 0x7fd1e40674b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.605+0000 7fd1fd4ee700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1f80cfd10 0x7fd1f815b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.605+0000 7fd1fd4ee700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1f80cb190 0x7fd1f815adb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.605+0000 7fd1fd4ee700 1 --2- 172.21.15.3:0/3573875882 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7fd1f806db10 0x7fd1f8155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.605+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3573875882 >> 172.21.15.3:0/3573875882 conn(0x7fd1f80ccc40 msgr2=0x7fd1f80b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:32.616 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.605+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3573875882 shutdown_connections 2023-03-03T06:38:32.617 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.605+0000 7fd1fd4ee700 1 -- 172.21.15.3:0/3573875882 wait complete. 2023-03-03T06:38:32.650 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:19", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:32.650 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-03-03T00:00:00', 'created': '2023-03-03T06:27:19', 'first': '2023-03-03T06:28:00', 'last': '2023-03-03T06:38:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-03-03T06:38:32.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:32.651 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-03-03T06:38:32.684 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_28_01_UTC 2023-03-03T06:38:32.684 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_29_00_UTC 2023-03-03T06:38:32.684 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_30_00_UTC 2023-03-03T06:38:32.685 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_31_00_UTC 2023-03-03T06:38:32.685 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_32_00_UTC 2023-03-03T06:38:32.685 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_33_00_UTC 2023-03-03T06:38:32.685 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_34_00_UTC 2023-03-03T06:38:32.685 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_35_00_UTC 2023-03-03T06:38:32.685 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_36_00_UTC 2023-03-03T06:38:32.685 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_37_00_UTC 2023-03-03T06:38:32.686 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_38_01_UTC 2023-03-03T06:38:32.687 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-03-03-06_28_01_UTC', 'scheduled-2023-03-03-06_29_00_UTC', 'scheduled-2023-03-03-06_30_00_UTC', 'scheduled-2023-03-03-06_31_00_UTC', 'scheduled-2023-03-03-06_32_00_UTC', 'scheduled-2023-03-03-06_33_00_UTC', 'scheduled-2023-03-03-06_34_00_UTC', 'scheduled-2023-03-03-06_35_00_UTC', 'scheduled-2023-03-03-06_36_00_UTC', 'scheduled-2023-03-03-06_37_00_UTC', 'scheduled-2023-03-03-06_38_01_UTC'] 2023-03-03T06:38:32.687 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir8 2023-03-03T06:38:32.932 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.921+0000 7efcf3328700 1 Processor -- start 2023-03-03T06:38:32.933 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.921+0000 7efcf3328700 1 -- start start 2023-03-03T06:38:32.935 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf3328700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efcec1595a0 0x7efcec06c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:32.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf3328700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcec0b07c0 0x7efcec0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:32.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf3328700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcec06c8c0 con 0x7efcec06d360 2023-03-03T06:38:32.936 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf3328700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcec0b1140 con 0x7efcec1595a0 2023-03-03T06:38:32.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf3328700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcec06d730 con 0x7efcec0b07c0 2023-03-03T06:38:32.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf1b25700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efcec1595a0 0x7efcec06c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:32.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf1b25700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efcec1595a0 0x7efcec06c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38600/0 (socket says 172.21.15.3:38600) 2023-03-03T06:38:32.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf1b25700 1 -- 172.21.15.3:0/1350134981 learned_addr learned my addr 172.21.15.3:0/1350134981 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:32.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf1b25700 1 -- 172.21.15.3:0/1350134981 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcec0b07c0 msgr2=0x7efcec0b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:32.937 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf2b27700 1 --2- 172.21.15.3:0/1350134981 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcec0b07c0 0x7efcec0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:32.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf1b25700 1 --2- 172.21.15.3:0/1350134981 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcec0b07c0 0x7efcec0b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf1b25700 1 -- 172.21.15.3:0/1350134981 >> v1:172.21.15.32:6789/0 conn(0x7efcec06d360 legacy=0x7efcec0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:32.938 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf1b25700 1 -- 172.21.15.3:0/1350134981 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec15a6b0 con 0x7efcec1595a0 2023-03-03T06:38:32.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf2b27700 1 --2- 172.21.15.3:0/1350134981 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcec0b07c0 0x7efcec0b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-03T06:38:32.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf1b25700 1 --2- 172.21.15.3:0/1350134981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcec1595a0 0x7efcec06c380 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009910 tx=0x7efcdc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5418a035e91b4e78 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:32.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.925+0000 7efcf1324700 1 -- 172.21.15.3:0/1350134981 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc004230 con 0x7efcec1595a0 2023-03-03T06:38:32.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.929+0000 7efcf1324700 1 -- 172.21.15.3:0/1350134981 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcdc004390 con 0x7efcec1595a0 2023-03-03T06:38:32.939 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.929+0000 7efcf1324700 1 -- 172.21.15.3:0/1350134981 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc009e60 con 0x7efcec1595a0 2023-03-03T06:38:32.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.929+0000 7efcf3328700 1 -- 172.21.15.3:0/1350134981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcec1595a0 msgr2=0x7efcec06c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:32.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.929+0000 7efcf3328700 1 --2- 172.21.15.3:0/1350134981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcec1595a0 0x7efcec06c380 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009910 tx=0x7efcdc009c20 comp rx=0 tx=0).stop 2023-03-03T06:38:32.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.929+0000 7efcf3328700 1 -- 172.21.15.3:0/1350134981 shutdown_connections 2023-03-03T06:38:32.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.929+0000 7efcf3328700 1 --2- 172.21.15.3:0/1350134981 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcec0b07c0 0x7efcec0b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.940 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.929+0000 7efcf3328700 1 --2- 172.21.15.3:0/1350134981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcec1595a0 0x7efcec06c380 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.929+0000 7efcf3328700 1 -- 172.21.15.3:0/1350134981 >> 172.21.15.3:0/1350134981 conn(0x7efcec0607c0 msgr2=0x7efcec060bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:32.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.929+0000 7efcf3328700 1 -- 172.21.15.3:0/1350134981 shutdown_connections 2023-03-03T06:38:32.941 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.929+0000 7efcf3328700 1 -- 172.21.15.3:0/1350134981 wait complete. 2023-03-03T06:38:32.942 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf3328700 1 Processor -- start 2023-03-03T06:38:32.943 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf3328700 1 -- start start 2023-03-03T06:38:32.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf3328700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec06d360 0x7efcec0c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:32.944 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf3328700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcec0b07c0 0x7efcec155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:32.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf3328700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcec1595a0 0x7efcec0c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:32.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf3328700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec0c8710 con 0x7efcec1595a0 2023-03-03T06:38:32.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf3328700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec155a70 con 0x7efcec06d360 2023-03-03T06:38:32.945 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf3328700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcec155d50 con 0x7efcec0b07c0 2023-03-03T06:38:32.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf2b27700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcec1595a0 0x7efcec0c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:32.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf2b27700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcec1595a0 0x7efcec0c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38612/0 (socket says 172.21.15.3:38612) 2023-03-03T06:38:32.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf2b27700 1 -- 172.21.15.3:0/1359849369 learned_addr learned my addr 172.21.15.3:0/1359849369 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:32.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf2326700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec06d360 0x7efcec0c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:32.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf1b25700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcec0b07c0 0x7efcec155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:32.947 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf2b27700 1 -- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcec0b07c0 msgr2=0x7efcec155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:32.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf2b27700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcec0b07c0 0x7efcec155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf2b27700 1 -- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec06d360 msgr2=0x7efcec0c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:32.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf2b27700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec06d360 0x7efcec0c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:32.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf2b27700 1 -- 172.21.15.3:0/1359849369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc009570 con 0x7efcec1595a0 2023-03-03T06:38:32.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.933+0000 7efcf1b25700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcec0b07c0 0x7efcec155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-03T06:38:32.948 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.937+0000 7efcf2b27700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcec1595a0 0x7efcec0c9910 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efce400cef0 tx=0x7efce400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:32.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.937+0000 7efce2ffd700 1 -- 172.21.15.3:0/1359849369 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce400ae50 con 0x7efcec1595a0 2023-03-03T06:38:32.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.937+0000 7efce2ffd700 1 -- 172.21.15.3:0/1359849369 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce4015440 con 0x7efcec1595a0 2023-03-03T06:38:32.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.937+0000 7efcf3328700 1 -- 172.21.15.3:0/1359849369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcec159e10 con 0x7efcec1595a0 2023-03-03T06:38:32.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.937+0000 7efce2ffd700 1 -- 172.21.15.3:0/1359849369 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce40157f0 con 0x7efcec1595a0 2023-03-03T06:38:32.949 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.937+0000 7efcf3328700 1 -- 172.21.15.3:0/1359849369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcec159b30 con 0x7efcec1595a0 2023-03-03T06:38:32.950 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.937+0000 7efcf3328700 1 -- 172.21.15.3:0/1359849369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcec15a0f0 con 0x7efcec1595a0 2023-03-03T06:38:32.951 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.941+0000 7efce2ffd700 1 -- 172.21.15.3:0/1359849369 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7efce401d360 con 0x7efcec1595a0 2023-03-03T06:38:32.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.941+0000 7efce2ffd700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efcd0065030 0x7efcd00674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:32.953 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.941+0000 7efce2ffd700 1 -- 172.21.15.3:0/1359849369 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7efce40825b0 con 0x7efcec1595a0 2023-03-03T06:38:32.954 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.945+0000 7efcf2326700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efcd0065030 0x7efcd00674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:32.955 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.945+0000 7efcf2326700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efcd0065030 0x7efcd00674e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcec0ce9f0 tx=0x7efce8009040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:32.957 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:32.945+0000 7efce2ffd700 1 -- 172.21.15.3:0/1359849369 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7efce40502b0 con 0x7efcec1595a0 2023-03-03T06:38:33.159 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.149+0000 7efcf3328700 1 -- 172.21.15.3:0/1359849369 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efcec073e90 con 0x7efcd0065030 2023-03-03T06:38:33.177 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.165+0000 7efce2ffd700 1 -- 172.21.15.3:0/1359849369 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7efcec073e90 con 0x7efcd0065030 2023-03-03T06:38:33.177 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:38:33.178 INFO:teuthology.orchestra.run.smithi003.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:20", "first": "2023-03-03T06:28:01", "last": "2023-03-03T06:38:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:33.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.169+0000 7efcf3328700 1 -- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efcd0065030 msgr2=0x7efcd00674e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:33.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.169+0000 7efcf3328700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efcd0065030 0x7efcd00674e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcec0ce9f0 tx=0x7efce8009040 comp rx=0 tx=0).stop 2023-03-03T06:38:33.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.169+0000 7efcf3328700 1 -- 172.21.15.3:0/1359849369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcec1595a0 msgr2=0x7efcec0c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:33.180 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.169+0000 7efcf3328700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcec1595a0 0x7efcec0c9910 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efce400cef0 tx=0x7efce400a340 comp rx=0 tx=0).stop 2023-03-03T06:38:33.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.169+0000 7efcf3328700 1 -- 172.21.15.3:0/1359849369 shutdown_connections 2023-03-03T06:38:33.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.169+0000 7efcf3328700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7efcd0065030 0x7efcd00674e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.169+0000 7efcf3328700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcec1595a0 0x7efcec0c9910 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.169+0000 7efcf3328700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7efcec0b07c0 0x7efcec155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.169+0000 7efcf3328700 1 --2- 172.21.15.3:0/1359849369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcec06d360 0x7efcec0c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.181 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.169+0000 7efcf3328700 1 -- 172.21.15.3:0/1359849369 >> 172.21.15.3:0/1359849369 conn(0x7efcec0607c0 msgr2=0x7efcec1589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:33.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.169+0000 7efcf3328700 1 -- 172.21.15.3:0/1359849369 shutdown_connections 2023-03-03T06:38:33.182 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.169+0000 7efcf3328700 1 -- 172.21.15.3:0/1359849369 wait complete. 2023-03-03T06:38:33.211 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:20", "first": "2023-03-03T06:28:01", "last": "2023-03-03T06:38:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:33.211 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-03-03T00:00:00', 'created': '2023-03-03T06:27:20', 'first': '2023-03-03T06:28:01', 'last': '2023-03-03T06:38:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-03-03T06:38:33.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:33.212 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-03-03T06:38:33.241 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_28_00_UTC 2023-03-03T06:38:33.241 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_29_01_UTC 2023-03-03T06:38:33.241 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_30_00_UTC 2023-03-03T06:38:33.242 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_31_00_UTC 2023-03-03T06:38:33.242 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_32_00_UTC 2023-03-03T06:38:33.242 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_33_00_UTC 2023-03-03T06:38:33.242 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_34_00_UTC 2023-03-03T06:38:33.242 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_35_00_UTC 2023-03-03T06:38:33.242 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_36_00_UTC 2023-03-03T06:38:33.242 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_37_00_UTC 2023-03-03T06:38:33.242 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_38_00_UTC 2023-03-03T06:38:33.244 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-03-03-06_28_00_UTC', 'scheduled-2023-03-03-06_29_01_UTC', 'scheduled-2023-03-03-06_30_00_UTC', 'scheduled-2023-03-03-06_31_00_UTC', 'scheduled-2023-03-03-06_32_00_UTC', 'scheduled-2023-03-03-06_33_00_UTC', 'scheduled-2023-03-03-06_34_00_UTC', 'scheduled-2023-03-03-06_35_00_UTC', 'scheduled-2023-03-03-06_36_00_UTC', 'scheduled-2023-03-03-06_37_00_UTC', 'scheduled-2023-03-03-06_38_00_UTC'] 2023-03-03T06:38:33.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir9 2023-03-03T06:38:33.512 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.501+0000 7f9591edc700 1 Processor -- start 2023-03-03T06:38:33.513 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.501+0000 7f9591edc700 1 -- start start 2023-03-03T06:38:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.505+0000 7f9591edc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 0x7f958c071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:33.516 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.505+0000 7f9591edc700 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f958c15d6e0 0x7f958c0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.505+0000 7f9591edc700 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f958c0cf3b0 con 0x7f958c0cfd80 2023-03-03T06:38:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.505+0000 7f9591edc700 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f958c156f70 con 0x7f958c15d6e0 2023-03-03T06:38:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.505+0000 7f9591edc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958c0cd320 con 0x7f958c156ba0 2023-03-03T06:38:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.505+0000 7f9590eda700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 0x7f958c071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.509+0000 7f9590eda700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 0x7f958c071280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:52426/0 (socket says 172.21.15.3:52426) 2023-03-03T06:38:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.509+0000 7f9590eda700 1 -- 172.21.15.3:0/1515535299 learned_addr learned my addr 172.21.15.3:0/1515535299 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.509+0000 7f958bfff700 1 --2- 172.21.15.3:0/1515535299 >> v2:172.21.15.32:3300/0 conn(0x7f958c15d6e0 0x7f958c0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.509+0000 7f9590eda700 1 -- 172.21.15.3:0/1515535299 >> v2:172.21.15.32:3300/0 conn(0x7f958c15d6e0 msgr2=0x7f958c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.509+0000 7f9590eda700 1 --2- 172.21.15.3:0/1515535299 >> v2:172.21.15.32:3300/0 conn(0x7f958c15d6e0 0x7f958c0cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.509+0000 7f9590eda700 1 -- 172.21.15.3:0/1515535299 >> v1:172.21.15.32:6789/0 conn(0x7f958c0cfd80 legacy=0x7f958c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:33.519 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.509+0000 7f9590eda700 1 -- 172.21.15.3:0/1515535299 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c0cd600 con 0x7f958c156ba0 2023-03-03T06:38:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.509+0000 7f9590eda700 1 --2- 172.21.15.3:0/1515535299 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 0x7f958c071280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9580009910 tx=0x7f9580009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf1d6d064d4ea89b server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.509+0000 7f958b7fe700 1 -- 172.21.15.3:0/1515535299 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580011830 con 0x7f958c156ba0 2023-03-03T06:38:33.520 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.509+0000 7f958b7fe700 1 -- 172.21.15.3:0/1515535299 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9580011990 con 0x7f958c156ba0 2023-03-03T06:38:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.509+0000 7f958b7fe700 1 -- 172.21.15.3:0/1515535299 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580011d40 con 0x7f958c156ba0 2023-03-03T06:38:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.509+0000 7f9591edc700 1 -- 172.21.15.3:0/1515535299 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 msgr2=0x7f958c071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.509+0000 7f9591edc700 1 --2- 172.21.15.3:0/1515535299 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 0x7f958c071280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9580009910 tx=0x7f9580009c20 comp rx=0 tx=0).stop 2023-03-03T06:38:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.513+0000 7f9591edc700 1 -- 172.21.15.3:0/1515535299 shutdown_connections 2023-03-03T06:38:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.513+0000 7f9591edc700 1 --2- 172.21.15.3:0/1515535299 >> v2:172.21.15.32:3300/0 conn(0x7f958c15d6e0 0x7f958c0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.513+0000 7f9591edc700 1 --2- 172.21.15.3:0/1515535299 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 0x7f958c071280 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.513+0000 7f9591edc700 1 -- 172.21.15.3:0/1515535299 >> 172.21.15.3:0/1515535299 conn(0x7f958c06e5c0 msgr2=0x7f958c06e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:33.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.513+0000 7f9591edc700 1 -- 172.21.15.3:0/1515535299 shutdown_connections 2023-03-03T06:38:33.523 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.513+0000 7f9591edc700 1 -- 172.21.15.3:0/1515535299 wait complete. 2023-03-03T06:38:33.525 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.513+0000 7f9591edc700 1 Processor -- start 2023-03-03T06:38:33.526 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.517+0000 7f9591edc700 1 -- start start 2023-03-03T06:38:33.528 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.517+0000 7f9591edc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f958c0cfd80 0x7f958c0c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:33.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.517+0000 7f9591edc700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 0x7f958c0c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:33.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.517+0000 7f9591edc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c15d6e0 0x7f958c0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:33.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.517+0000 7f9591edc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c0c9600 con 0x7f958c15d6e0 2023-03-03T06:38:33.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.517+0000 7f9591edc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c0b2670 con 0x7f958c0cfd80 2023-03-03T06:38:33.529 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.517+0000 7f9591edc700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958c0b2950 con 0x7f958c156ba0 2023-03-03T06:38:33.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.517+0000 7f958bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 0x7f958c0c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:33.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.517+0000 7f958bfff700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 0x7f958c0c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3301/0 says I am v2:172.21.15.3:52434/0 (socket says 172.21.15.3:52434) 2023-03-03T06:38:33.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.517+0000 7f958bfff700 1 -- 172.21.15.3:0/784280552 learned_addr learned my addr 172.21.15.3:0/784280552 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:33.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.517+0000 7f95916db700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c15d6e0 0x7f958c0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:33.530 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.517+0000 7f9590eda700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f958c0cfd80 0x7f958c0c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:33.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.521+0000 7f958bfff700 1 -- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f958c0cfd80 msgr2=0x7f958c0c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:33.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.521+0000 7f958bfff700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f958c0cfd80 0x7f958c0c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.521+0000 7f958bfff700 1 -- 172.21.15.3:0/784280552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c15d6e0 msgr2=0x7f958c0c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:33.531 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.521+0000 7f958bfff700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c15d6e0 0x7f958c0c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.521+0000 7f958bfff700 1 -- 172.21.15.3:0/784280552 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9580009570 con 0x7f958c156ba0 2023-03-03T06:38:33.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.521+0000 7f958bfff700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 0x7f958c0c90c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f957800e9d0 tx=0x7f957800ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:33.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.521+0000 7f95897fa700 1 -- 172.21.15.3:0/784280552 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578009720 con 0x7f958c156ba0 2023-03-03T06:38:33.532 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.521+0000 7f95897fa700 1 -- 172.21.15.3:0/784280552 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957800ee70 con 0x7f958c156ba0 2023-03-03T06:38:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.521+0000 7f95897fa700 1 -- 172.21.15.3:0/784280552 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957800ca50 con 0x7f958c156ba0 2023-03-03T06:38:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.521+0000 7f9591edc700 1 -- 172.21.15.3:0/784280552 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958c0717c0 con 0x7f958c156ba0 2023-03-03T06:38:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.521+0000 7f9591edc700 1 -- 172.21.15.3:0/784280552 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958c0cd8e0 con 0x7f958c156ba0 2023-03-03T06:38:33.534 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.525+0000 7f9591edc700 1 -- 172.21.15.3:0/784280552 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958c0cef40 con 0x7f958c156ba0 2023-03-03T06:38:33.538 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.529+0000 7f95897fa700 1 -- 172.21.15.3:0/784280552 <== mon.2 v2:172.21.15.32:3301/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f9578019030 con 0x7f958c156ba0 2023-03-03T06:38:33.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.529+0000 7f95897fa700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9570065050 0x7f9570067500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:33.540 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.529+0000 7f95897fa700 1 -- 172.21.15.3:0/784280552 <== mon.2 v2:172.21.15.32:3301/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f95780821f0 con 0x7f958c156ba0 2023-03-03T06:38:33.541 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.529+0000 7f9590eda700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9570065050 0x7f9570067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:33.542 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.533+0000 7f9590eda700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9570065050 0x7f9570067500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9580000c00 tx=0x7f9580008030 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:33.550 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.541+0000 7f95897fa700 1 -- 172.21.15.3:0/784280552 <== mon.2 v2:172.21.15.32:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f957804ffd0 con 0x7f958c156ba0 2023-03-03T06:38:33.757 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.745+0000 7f9591edc700 1 -- 172.21.15.3:0/784280552 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f958c073e90 con 0x7f9570065050 2023-03-03T06:38:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.769+0000 7f95897fa700 1 -- 172.21.15.3:0/784280552 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f958c073e90 con 0x7f9570065050 2023-03-03T06:38:33.779 INFO:teuthology.orchestra.run.smithi003.stdout: 2023-03-03T06:38:33.780 INFO:teuthology.orchestra.run.smithi003.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:21", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:33.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.773+0000 7f9591edc700 1 -- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9570065050 msgr2=0x7f9570067500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:33.782 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.773+0000 7f9591edc700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9570065050 0x7f9570067500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9580000c00 tx=0x7f9580008030 comp rx=0 tx=0).stop 2023-03-03T06:38:33.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.773+0000 7f9591edc700 1 -- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 msgr2=0x7f958c0c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:33.783 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.773+0000 7f9591edc700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 0x7f958c0c90c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f957800e9d0 tx=0x7f957800ece0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.773+0000 7f9591edc700 1 -- 172.21.15.3:0/784280552 shutdown_connections 2023-03-03T06:38:33.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.773+0000 7f9591edc700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f9570065050 0x7f9570067500 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.784 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.773+0000 7f9591edc700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c15d6e0 0x7f958c0c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.773+0000 7f9591edc700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f958c156ba0 0x7f958c0c90c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.773+0000 7f9591edc700 1 --2- 172.21.15.3:0/784280552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f958c0cfd80 0x7f958c0c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:33.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.773+0000 7f9591edc700 1 -- 172.21.15.3:0/784280552 >> 172.21.15.3:0/784280552 conn(0x7f958c06e5c0 msgr2=0x7f958c15ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:33.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.773+0000 7f9591edc700 1 -- 172.21.15.3:0/784280552 shutdown_connections 2023-03-03T06:38:33.785 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:33.773+0000 7f9591edc700 1 -- 172.21.15.3:0/784280552 wait complete. 2023-03-03T06:38:33.819 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-03-03T00:00:00", "created": "2023-03-03T06:27:21", "first": "2023-03-03T06:28:00", "last": "2023-03-03T06:38:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-03-03T06:38:33.819 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-03-03T00:00:00', 'created': '2023-03-03T06:27:21', 'first': '2023-03-03T06:28:00', 'last': '2023-03-03T06:38:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-03-03T06:38:33.820 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir0 1M 2023-03-03T06:38:34.040 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.029+0000 7f5d991e8700 1 Processor -- start 2023-03-03T06:38:34.041 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.029+0000 7f5d991e8700 1 -- start start 2023-03-03T06:38:34.044 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.033+0000 7f5d991e8700 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d9415d6e0 0x7f5d940cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:34.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.033+0000 7f5d991e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d940cfd80 0x7f5d940d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:34.045 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.033+0000 7f5d991e8700 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d940cf3b0 con 0x7f5d94156ba0 2023-03-03T06:38:34.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.033+0000 7f5d991e8700 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d94156f70 con 0x7f5d9415d6e0 2023-03-03T06:38:34.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.033+0000 7f5d991e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940cd320 con 0x7f5d940cfd80 2023-03-03T06:38:34.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.033+0000 7f5d93fff700 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5d94156ba0 0x7f5d94071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51078/0 (socket says 172.21.15.3:51078) 2023-03-03T06:38:34.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.033+0000 7f5d93fff700 1 -- 172.21.15.3:0/3709103749 learned_addr learned my addr 172.21.15.3:0/3709103749 (peer_addr_for_me v1:172.21.15.3:0/0) 2023-03-03T06:38:34.046 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.033+0000 7f5d989e7700 1 --2- 172.21.15.3:0/3709103749 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d940cfd80 0x7f5d940d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:34.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.033+0000 7f5d937fe700 1 --2- 172.21.15.3:0/3709103749 >> v2:172.21.15.3:3300/0 conn(0x7f5d9415d6e0 0x7f5d940cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:34.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d989e7700 1 -- 172.21.15.3:0/3709103749 >> v2:172.21.15.3:3300/0 conn(0x7f5d9415d6e0 msgr2=0x7f5d940cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-03T06:38:34.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d989e7700 1 --2- 172.21.15.3:0/3709103749 >> v2:172.21.15.3:3300/0 conn(0x7f5d9415d6e0 0x7f5d940cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:34.047 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d989e7700 1 -- 172.21.15.3:0/3709103749 >> v1:172.21.15.3:6789/0 conn(0x7f5d94156ba0 legacy=0x7f5d94071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:34.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d989e7700 1 -- 172.21.15.3:0/3709103749 --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940cd600 con 0x7f5d940cfd80 2023-03-03T06:38:34.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d989e7700 1 --2- 172.21.15.3:0/3709103749 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d940cfd80 0x7f5d940d0b30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009d90 tx=0x7f5d8c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fb6528bd6f2d8d5 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:34.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d92ffd700 1 -- 172.21.15.3:0/3709103749 <== mon.2 v2:172.21.15.32:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00e070 con 0x7f5d940cfd80 2023-03-03T06:38:34.048 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d92ffd700 1 -- 172.21.15.3:0/3709103749 <== mon.2 v2:172.21.15.32:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c00bbd0 con 0x7f5d940cfd80 2023-03-03T06:38:34.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d92ffd700 1 -- 172.21.15.3:0/3709103749 <== mon.2 v2:172.21.15.32:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c012430 con 0x7f5d940cfd80 2023-03-03T06:38:34.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d991e8700 1 -- 172.21.15.3:0/3709103749 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d940cfd80 msgr2=0x7f5d940d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:34.049 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d991e8700 1 --2- 172.21.15.3:0/3709103749 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d940cfd80 0x7f5d940d0b30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009d90 tx=0x7f5d8c00b330 comp rx=0 tx=0).stop 2023-03-03T06:38:34.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d991e8700 1 -- 172.21.15.3:0/3709103749 shutdown_connections 2023-03-03T06:38:34.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d991e8700 1 --2- 172.21.15.3:0/3709103749 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d940cfd80 0x7f5d940d0b30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:34.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d991e8700 1 --2- 172.21.15.3:0/3709103749 >> v2:172.21.15.3:3300/0 conn(0x7f5d9415d6e0 0x7f5d940cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:34.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d991e8700 1 -- 172.21.15.3:0/3709103749 >> 172.21.15.3:0/3709103749 conn(0x7f5d9406e5c0 msgr2=0x7f5d9406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:34.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d991e8700 1 -- 172.21.15.3:0/3709103749 shutdown_connections 2023-03-03T06:38:34.050 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.037+0000 7f5d991e8700 1 -- 172.21.15.3:0/3709103749 wait complete. 2023-03-03T06:38:34.051 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.041+0000 7f5d991e8700 1 Processor -- start 2023-03-03T06:38:34.052 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.041+0000 7f5d991e8700 1 -- start start 2023-03-03T06:38:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.041+0000 7f5d991e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d940cfd80 0x7f5d940b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.041+0000 7f5d991e8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d94156ba0 0x7f5d940c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.041+0000 7f5d991e8700 1 --2- >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d9415d6e0 0x7f5d9415e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.041+0000 7f5d991e8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940b4c10 con 0x7f5d94156ba0 2023-03-03T06:38:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.041+0000 7f5d991e8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940b36c0 con 0x7f5d940cfd80 2023-03-03T06:38:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.041+0000 7f5d991e8700 1 -- --> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940b4240 con 0x7f5d9415d6e0 2023-03-03T06:38:34.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d937fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d94156ba0 0x7f5d940c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:34.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d937fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d94156ba0 0x7f5d940c8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38624/0 (socket says 172.21.15.3:38624) 2023-03-03T06:38:34.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d937fe700 1 -- 172.21.15.3:0/634644485 learned_addr learned my addr 172.21.15.3:0/634644485 (peer_addr_for_me v2:172.21.15.3:0/0) 2023-03-03T06:38:34.055 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d989e7700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d9415d6e0 0x7f5d9415e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d93fff700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d940cfd80 0x7f5d940b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d937fe700 1 -- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d9415d6e0 msgr2=0x7f5d9415e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d937fe700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d9415d6e0 0x7f5d9415e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d937fe700 1 -- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d940cfd80 msgr2=0x7f5d940b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d937fe700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d940cfd80 0x7f5d940b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d937fe700 1 -- 172.21.15.3:0/634644485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c009a70 con 0x7f5d94156ba0 2023-03-03T06:38:34.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d937fe700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d94156ba0 0x7f5d940c8420 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800e790 tx=0x7f5d8800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:34.057 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d90ff9700 1 -- 172.21.15.3:0/634644485 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88013610 con 0x7f5d94156ba0 2023-03-03T06:38:34.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d90ff9700 1 -- 172.21.15.3:0/634644485 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d88013770 con 0x7f5d94156ba0 2023-03-03T06:38:34.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d90ff9700 1 -- 172.21.15.3:0/634644485 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88013ae0 con 0x7f5d94156ba0 2023-03-03T06:38:34.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d991e8700 1 -- 172.21.15.3:0/634644485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d940cef40 con 0x7f5d94156ba0 2023-03-03T06:38:34.058 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.045+0000 7f5d991e8700 1 -- 172.21.15.3:0/634644485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d940cd8e0 con 0x7f5d94156ba0 2023-03-03T06:38:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.049+0000 7f5d991e8700 1 -- 172.21.15.3:0/634644485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d940cf0d0 con 0x7f5d94156ba0 2023-03-03T06:38:34.060 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.049+0000 7f5d90ff9700 1 -- 172.21.15.3:0/634644485 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) v1 ==== 84096+0+0 (secure 0 0 0) 0x7f5d88023070 con 0x7f5d94156ba0 2023-03-03T06:38:34.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.053+0000 7f5d90ff9700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5d80065000 0x7f5d800674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-03T06:38:34.062 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.053+0000 7f5d90ff9700 1 -- 172.21.15.3:0/634644485 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6698+0+0 (secure 0 0 0) 0x7f5d88081ff0 con 0x7f5d94156ba0 2023-03-03T06:38:34.063 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.053+0000 7f5d93fff700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5d80065000 0x7f5d800674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-03T06:38:34.064 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.053+0000 7f5d93fff700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5d80065000 0x7f5d800674b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d84000f60 tx=0x7f5d84009040 comp rx=0 tx=0).ready entity=mgr.5494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-03T06:38:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.057+0000 7f5d90ff9700 1 -- 172.21.15.3:0/634644485 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5d880b2400 con 0x7f5d94156ba0 2023-03-03T06:38:34.264 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.253+0000 7f5d991e8700 1 -- 172.21.15.3:0/634644485 --> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d940615d0 con 0x7f5d80065000 2023-03-03T06:38:34.442 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.429+0000 7f5d90ff9700 1 -- 172.21.15.3:0/634644485 <== mgr.5494 v2:172.21.15.32:6832/32113 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f5d940615d0 con 0x7f5d80065000 2023-03-03T06:38:34.442 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-03-03T06:38:34.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.433+0000 7f5d991e8700 1 -- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5d80065000 msgr2=0x7f5d800674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:34.445 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.433+0000 7f5d991e8700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5d80065000 0x7f5d800674b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d84000f60 tx=0x7f5d84009040 comp rx=0 tx=0).stop 2023-03-03T06:38:34.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.433+0000 7f5d991e8700 1 -- 172.21.15.3:0/634644485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d94156ba0 msgr2=0x7f5d940c8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-03T06:38:34.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.433+0000 7f5d991e8700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d94156ba0 0x7f5d940c8420 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800e790 tx=0x7f5d8800eb50 comp rx=0 tx=0).stop 2023-03-03T06:38:34.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.433+0000 7f5d991e8700 1 -- 172.21.15.3:0/634644485 shutdown_connections 2023-03-03T06:38:34.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.433+0000 7f5d991e8700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:6832/32113,v1:172.21.15.32:6833/32113] conn(0x7f5d80065000 0x7f5d800674b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:34.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.433+0000 7f5d991e8700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:3301/0,v1:172.21.15.32:6790/0] conn(0x7f5d9415d6e0 0x7f5d9415e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:34.446 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.433+0000 7f5d991e8700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d94156ba0 0x7f5d940c8420 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:34.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.433+0000 7f5d991e8700 1 --2- 172.21.15.3:0/634644485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d940cfd80 0x7f5d940b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-03T06:38:34.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.433+0000 7f5d991e8700 1 -- 172.21.15.3:0/634644485 >> 172.21.15.3:0/634644485 conn(0x7f5d9406e5c0 msgr2=0x7f5d9415cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-03T06:38:34.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.437+0000 7f5d991e8700 1 -- 172.21.15.3:0/634644485 shutdown_connections 2023-03-03T06:38:34.447 INFO:teuthology.orchestra.run.smithi003.stderr:2023-03-03T06:38:34.437+0000 7f5d991e8700 1 -- 172.21.15.3:0/634644485 wait complete. 2023-03-03T06:38:34.480 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-03-03T06:38:34.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:34.481 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-03-03T06:38:34.506 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_28_00_UTC 2023-03-03T06:38:34.507 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_29_00_UTC 2023-03-03T06:38:34.507 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_30_00_UTC 2023-03-03T06:38:34.507 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_31_01_UTC 2023-03-03T06:38:34.507 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_32_00_UTC 2023-03-03T06:38:34.507 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_33_00_UTC 2023-03-03T06:38:34.507 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_34_00_UTC 2023-03-03T06:38:34.507 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_35_00_UTC 2023-03-03T06:38:34.508 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_36_01_UTC 2023-03-03T06:38:34.508 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_37_00_UTC 2023-03-03T06:38:34.508 INFO:teuthology.orchestra.run.smithi003.stdout:scheduled-2023-03-03-06_38_00_UTC 2023-03-03T06:38:34.509 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_28_00_UTC 2023-03-03T06:38:34.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:34.510 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_28_00_UTC) 2023-03-03T06:38:34.593 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_29_00_UTC 2023-03-03T06:38:34.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:34.594 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_29_00_UTC) 2023-03-03T06:38:34.679 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_30_00_UTC 2023-03-03T06:38:34.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:34.680 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_30_00_UTC) 2023-03-03T06:38:34.762 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_31_01_UTC 2023-03-03T06:38:34.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:34.763 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_31_01_UTC) 2023-03-03T06:38:34.852 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_32_00_UTC 2023-03-03T06:38:34.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:34.853 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_32_00_UTC) 2023-03-03T06:38:34.936 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_33_00_UTC 2023-03-03T06:38:34.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:34.937 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_33_00_UTC) 2023-03-03T06:38:35.023 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_34_00_UTC 2023-03-03T06:38:35.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:35.024 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_34_00_UTC) 2023-03-03T06:38:35.110 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_35_00_UTC 2023-03-03T06:38:35.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:35.111 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_35_00_UTC) 2023-03-03T06:38:35.197 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_36_01_UTC 2023-03-03T06:38:35.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:35.198 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_36_01_UTC) 2023-03-03T06:38:35.285 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_37_00_UTC 2023-03-03T06:38:35.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:35.286 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_37_00_UTC) 2023-03-03T06:38:35.367 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_38_00_UTC 2023-03-03T06:38:35.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:35.368 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-03-03-06_38_00_UTC) 2023-03-03T06:38:35.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-03T06:38:35.457 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-03-03T06:38:35.525 INFO:tasks.ceph.mds.b.smithi032.stderr:2023-03-03T06:38:35.521+0000 7fdbe1532700 -1 mds.0.cache.den(0x602 10000000001) newly corrupt dentry to be committed: [dentry #0x100/stray2/10000000001 [7b,head] auth NULL (dn xlock x=1 by 0x5613cb81c800) (dversion lock w=1 last_client=55034) pv=263 v=1 ap=2 ino=(nil) state=1342177281|new | request=1 lock=2 authpin=1 0x5613cbc6fb80] 2023-03-03T06:38:35.526 INFO:tasks.ceph.mds.b.smithi032.stderr:2023-03-03T06:38:35.521+0000 7fdbe1532700 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x602is fatal because it is a system directory for this rank 2023-03-03T06:38:36.122 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T06:38:36.109+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-03-03T06:38:36.122 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T06:38:36.109+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-03-03T06:38:36.167 INFO:tasks.ceph.mds.b.smithi032.stderr: -35> 2023-03-03T06:38:35.521+0000 7fdbe1532700 -1 mds.0.cache.den(0x602 10000000001) newly corrupt dentry to be committed: [dentry #0x100/stray2/10000000001 [7b,head] auth NULL (dn xlock x=1 by 0x5613cb81c800) (dversion lock w=1 last_client=55034) pv=263 v=1 ap=2 ino=(nil) state=1342177281|new | request=1 lock=2 authpin=1 0x5613cbc6fb80] 2023-03-03T06:38:36.168 INFO:tasks.ceph.mds.b.smithi032.stderr: -33> 2023-03-03T06:38:35.521+0000 7fdbe1532700 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x602is fatal because it is a system directory for this rank 2023-03-03T06:38:36.238 INFO:tasks.ceph.mds.b.smithi032.stdout:starting mds.b at 2023-03-03T06:38:59.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:38:59.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:39:29.767 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:39:29.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:39:59.879 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:39:59.887 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:40:00.006 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T06:39:59.992+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T06:40:29.968 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:40:29.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:41:00.057 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:41:00.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:41:30.146 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:41:30.153 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:42:00.227 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:42:00.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:42:30.328 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:42:30.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:43:00.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:43:00.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:43:30.514 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:43:30.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:43:35.510 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2023-03-03T06:43:35.512 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_teuthology_c7c3cc84dcd2bd1fdc61d0abde9057ccda2484b6/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2023-03-03T06:44:00.611 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:44:00.619 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:44:30.747 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:44:30.756 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:45:00.840 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:45:00.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:45:30.932 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:45:30.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:46:01.035 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:46:01.044 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:46:31.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:46:31.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:47:01.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:47:01.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:47:31.316 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:47:31.324 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:48:01.413 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:48:01.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:48:31.501 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:48:31.510 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:49:01.612 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:49:01.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:49:31.709 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:49:31.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:50:00.006 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T06:49:59.993+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T06:50:01.805 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:50:01.814 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:50:31.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:50:31.898 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:51:02.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:51:02.012 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:51:32.103 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:51:32.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:52:02.199 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:52:02.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:52:32.323 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:52:32.332 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:53:02.414 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:53:02.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:53:32.511 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:53:32.520 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:54:02.600 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:54:02.609 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:54:32.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:54:32.698 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:55:02.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:55:02.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:55:32.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:55:32.988 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:56:03.061 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:56:03.070 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:56:33.166 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:56:33.175 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:57:03.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:57:03.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:57:33.364 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:57:33.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:58:03.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:58:03.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:58:33.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:58:33.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:59:03.638 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:59:03.647 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:59:33.728 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T06:59:33.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:00:00.002 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T06:59:59.998+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T07:00:03.814 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:00:03.823 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:00:33.908 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:00:33.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:01:04.000 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:01:04.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:01:34.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:01:34.097 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:02:04.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:02:04.190 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:02:34.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:02:34.294 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:03:04.375 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:03:04.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:03:34.470 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:03:34.479 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:04:04.563 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:04:04.571 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:04:34.653 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:04:34.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:05:04.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:05:04.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:05:34.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:05:34.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:06:04.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:06:04.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:06:35.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:06:35.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:07:05.142 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:07:05.169 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:07:35.275 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:07:35.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:08:05.354 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:08:05.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:08:35.447 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:08:35.456 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:09:05.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:09:05.545 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:09:35.636 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:09:35.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:10:00.002 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T07:09:59.994+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T07:10:05.728 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:10:05.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:10:35.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:10:35.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:11:05.912 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:11:05.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:11:36.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:11:36.013 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:12:06.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:12:06.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:12:36.199 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:12:36.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:13:06.278 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:13:06.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:13:36.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:13:36.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:14:06.452 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:14:06.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:14:36.541 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:14:36.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:15:06.633 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:15:06.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:15:36.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:15:36.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:16:06.826 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:16:06.835 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:16:36.916 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:16:36.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:17:07.015 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:17:07.024 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:17:37.123 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:17:37.131 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:18:07.221 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:18:07.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:18:37.311 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:18:37.320 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:19:07.400 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:19:07.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:19:37.582 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:19:37.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:20:00.002 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T07:19:59.999+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T07:20:07.698 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:20:07.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:20:37.805 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:20:37.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:21:07.903 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:21:07.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:21:37.990 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:21:38.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:22:08.083 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:22:08.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:22:38.191 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:22:38.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:23:08.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:23:08.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:23:38.378 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:23:38.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:24:08.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:24:08.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:24:38.566 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:24:38.574 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:25:08.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:25:08.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:25:38.764 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:25:38.772 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:26:08.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:26:08.859 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:26:38.935 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:26:38.944 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:27:09.023 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:27:09.032 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:27:39.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:27:39.143 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:28:09.232 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:28:09.241 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:28:39.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:28:39.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:29:09.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:29:09.418 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:29:39.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:29:39.511 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:30:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T07:29:59.992+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T07:30:09.608 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:30:09.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:30:39.700 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:30:39.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:31:09.797 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:31:09.806 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:31:40.073 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:31:40.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:32:10.161 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:32:10.169 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:32:40.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:32:40.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:33:10.366 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:33:10.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:33:40.454 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:33:40.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:34:10.544 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:34:10.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:34:40.636 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:34:40.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:35:10.725 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:35:10.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:35:40.816 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:35:40.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:36:10.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:36:10.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:36:41.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:36:41.137 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:37:11.211 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:37:11.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:37:41.309 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:37:41.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:38:11.407 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:38:11.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:38:41.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:38:41.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:39:11.611 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:39:11.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:39:41.701 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:39:41.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:40:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T07:39:59.993+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T07:40:11.788 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:40:11.797 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:40:41.899 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:40:41.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:41:11.987 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:41:11.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:41:42.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:41:42.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:42:12.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:42:12.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:42:42.276 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:42:42.285 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:43:12.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:43:12.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:43:42.464 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:43:42.473 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:44:12.558 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:44:12.566 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:44:42.652 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:44:42.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:45:12.746 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:45:12.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:45:42.866 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:45:42.875 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:46:12.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:46:12.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:46:43.045 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:46:43.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:47:13.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:47:13.159 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:47:43.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:47:43.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:48:13.338 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:48:13.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:48:43.430 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:48:43.438 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:49:13.515 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:49:13.524 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:49:43.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:49:43.618 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:50:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T07:49:59.998+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T07:50:13.707 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:50:13.716 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:50:43.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:50:43.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:51:13.894 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:51:13.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:51:43.986 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:51:43.995 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:52:14.083 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:52:14.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:52:44.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:52:44.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:53:14.307 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:53:14.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:53:44.405 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:53:44.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:54:14.495 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:54:14.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:54:44.592 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:54:44.600 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:55:14.680 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:55:14.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:55:44.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:55:44.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:56:14.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:56:14.871 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:56:44.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:56:44.961 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:57:15.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:57:15.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:57:45.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:57:45.157 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:58:15.238 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:58:15.247 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:58:45.347 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:58:45.357 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:59:15.436 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:59:15.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:59:45.526 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T07:59:45.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:00:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T07:59:59.994+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T08:00:15.625 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:00:15.634 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:00:45.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:00:45.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:01:15.821 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:01:15.829 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:01:45.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:01:45.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:02:15.993 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:02:16.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:02:46.099 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:02:46.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:03:16.199 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:03:16.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:03:46.294 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:03:46.308 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:04:16.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:04:16.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:04:46.498 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:04:46.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:05:16.581 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:05:16.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:05:46.670 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:05:46.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:06:16.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:06:16.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:06:46.859 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:06:46.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:07:16.959 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:07:16.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:07:47.066 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:07:47.074 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:08:17.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:08:17.161 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:08:47.237 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:08:47.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:09:17.330 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:09:17.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:09:47.427 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:09:47.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:10:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T08:09:59.998+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T08:10:17.533 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:10:17.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:10:47.630 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:10:47.638 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:11:17.719 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:11:17.726 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:11:47.809 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:11:47.818 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:12:17.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:12:17.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:12:47.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:12:48.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:13:18.105 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:13:18.114 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:13:48.204 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:13:48.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:14:18.302 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:14:18.310 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:14:48.391 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:14:48.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:15:18.492 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:15:18.502 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:15:48.592 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:15:48.600 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:16:18.686 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:16:18.694 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:16:48.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:16:48.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:17:18.868 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:17:18.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:17:48.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:17:48.978 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:18:19.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:18:19.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:18:49.162 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:18:49.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:19:19.259 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:19:19.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:19:49.358 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:19:49.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:20:00.002 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T08:19:59.994+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T08:20:19.450 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:20:19.459 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:20:49.539 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:20:49.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:21:19.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:21:19.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:21:49.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:21:49.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:22:19.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:22:19.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:22:49.951 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:22:49.960 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:23:20.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:23:20.059 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:23:50.147 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:23:50.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:24:20.229 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:24:20.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:24:50.308 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:24:50.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:25:20.423 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:25:20.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:25:50.508 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:25:50.519 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:26:20.604 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:26:20.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:26:50.701 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:26:50.708 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:27:20.793 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:27:20.801 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:27:50.901 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:27:50.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:28:20.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:28:20.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:28:51.088 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:28:51.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:29:21.194 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:29:21.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:29:51.280 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:29:51.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:30:00.002 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T08:29:59.999+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T08:30:21.388 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:30:21.396 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:30:51.492 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:30:51.502 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:31:21.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:31:21.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:31:51.680 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:31:51.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:32:21.793 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:32:21.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:32:51.912 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:32:51.920 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:33:22.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:33:22.012 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:33:52.104 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:33:52.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:34:22.192 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:34:22.201 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:34:52.300 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:34:52.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:35:22.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:35:22.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:35:52.497 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:35:52.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:36:22.597 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:36:22.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:36:52.693 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:36:52.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:37:22.783 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:37:22.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:37:52.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:37:52.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:38:23.010 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:38:23.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:38:53.119 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:38:53.128 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:39:23.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:39:23.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:39:53.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:39:53.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:40:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T08:39:59.999+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T08:40:23.397 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:40:23.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:40:53.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:40:53.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:41:23.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:41:23.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:41:53.716 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:41:53.726 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:42:23.807 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:42:23.816 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:42:53.911 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:42:53.920 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:43:23.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:43:24.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:43:54.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:43:54.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:44:24.183 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:44:24.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:44:54.266 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:44:54.274 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:45:24.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:45:24.356 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:45:54.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:45:54.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:46:24.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:46:24.545 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:46:54.624 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:46:54.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:47:24.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:47:24.732 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:47:54.875 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:47:54.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:48:25.007 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:48:25.016 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:48:55.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:48:55.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:49:25.184 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:49:25.194 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:49:55.291 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:49:55.298 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:50:00.002 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T08:49:59.996+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T08:50:25.376 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:50:25.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:50:55.474 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:50:55.483 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:51:25.573 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:51:25.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:51:55.666 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:51:55.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:52:25.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:52:25.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:52:55.882 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:52:55.891 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:53:25.981 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:53:25.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:53:56.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:53:56.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:54:26.179 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:54:26.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:54:56.277 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:54:56.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:55:26.392 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:55:26.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:55:56.481 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:55:56.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:56:26.581 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:56:26.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:56:56.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:56:56.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:57:26.790 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:57:26.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:57:56.908 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:57:56.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:58:26.998 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:58:27.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:58:57.097 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:58:57.106 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:59:27.188 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:59:27.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:59:57.284 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T08:59:57.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:00:00.002 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T08:59:59.997+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T09:00:27.381 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:00:27.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:00:57.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:00:57.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:01:27.570 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:01:27.579 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:01:57.671 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:01:57.679 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:02:27.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:02:27.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:02:58.000 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:02:58.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:03:28.100 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:03:28.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:03:58.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:03:58.203 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:04:28.302 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:04:28.311 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:04:58.409 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:04:58.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:05:28.500 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:05:28.509 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:05:58.585 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:05:58.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:06:28.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:06:28.699 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:06:58.801 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:06:58.809 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:07:28.900 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:07:28.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:07:59.010 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:07:59.019 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:08:29.100 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:08:29.109 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:08:59.190 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:08:59.199 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:09:29.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:09:29.304 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:09:59.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:09:59.393 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:10:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T09:09:59.997+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T09:10:29.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:10:29.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:10:59.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:10:59.668 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:11:29.742 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:11:29.750 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:11:59.838 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:11:59.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:12:29.933 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:12:29.942 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:13:00.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:13:00.035 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:13:30.122 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:13:30.131 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:14:00.206 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:14:00.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:14:30.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:14:30.308 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:15:00.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:15:00.393 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:15:30.486 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:15:30.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:16:00.585 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:16:00.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:16:30.680 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:16:30.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:17:00.754 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:17:00.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:17:30.917 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:17:30.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:18:01.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:18:01.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:18:31.145 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:18:31.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:19:01.245 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:19:01.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:19:31.331 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:19:31.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:20:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T09:19:59.994+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T09:20:01.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:20:01.433 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:20:31.525 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:20:31.533 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:21:01.613 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:21:01.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:21:31.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:21:31.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:22:01.793 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:22:01.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:22:31.892 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:22:31.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:23:01.983 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:23:01.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:23:32.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:23:32.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:24:02.179 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:24:02.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:24:32.252 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:24:32.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:25:02.337 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:25:02.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:25:32.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:25:32.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:26:02.563 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:26:02.571 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:26:32.660 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:26:32.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:27:02.758 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:27:02.767 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:27:32.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:27:32.866 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:28:02.956 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:28:02.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:28:33.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:28:33.062 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:29:03.143 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:29:03.151 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:29:33.237 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:29:33.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:30:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T09:29:59.999+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T09:30:03.330 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:30:03.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:30:33.427 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:30:33.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:31:03.534 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:31:03.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:31:33.626 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:31:33.634 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:32:03.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:32:03.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:32:33.833 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:32:33.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:33:03.946 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:33:03.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:33:34.035 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:33:34.044 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:34:04.205 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:34:04.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:34:34.309 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:34:34.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:35:04.394 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:35:04.402 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:35:34.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:35:34.498 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:36:04.582 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:36:04.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:36:34.698 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:36:34.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:37:04.787 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:37:04.796 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:37:34.888 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:37:34.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:38:04.991 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:38:05.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:38:35.117 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:38:35.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:39:05.205 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:39:05.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:39:35.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:39:35.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:40:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T09:39:59.999+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T09:40:05.367 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:40:05.375 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:40:35.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:40:35.473 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:41:05.547 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:41:05.556 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:41:35.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:41:35.654 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:42:05.736 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:42:05.744 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:42:35.824 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:42:35.833 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:43:05.924 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:43:05.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:43:36.004 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:43:36.013 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:44:06.104 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:44:06.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:44:36.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:44:36.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:45:06.299 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:45:06.307 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:45:36.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:45:36.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:46:06.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:46:06.496 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:46:36.584 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:46:36.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:47:06.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:47:06.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:47:36.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:47:36.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:48:06.881 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:48:06.890 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:48:36.978 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:48:36.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:49:07.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:49:07.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:49:37.175 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:49:37.183 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:50:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T09:49:59.996+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T09:50:07.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:50:07.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:50:37.364 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:50:37.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:51:07.454 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:51:07.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:51:37.558 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:51:37.568 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:52:07.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:52:07.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:52:37.754 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:52:37.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:53:07.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:53:07.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:53:37.951 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:53:37.958 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:54:08.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:54:08.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:54:38.140 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:54:38.149 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:55:08.241 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:55:08.250 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:55:38.335 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:55:38.344 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:56:08.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:56:08.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:56:38.532 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:56:38.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:57:08.629 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:57:08.638 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:57:38.728 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:57:38.737 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:58:08.826 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:58:08.835 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:58:38.941 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:58:38.950 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:59:09.024 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:59:09.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:59:39.123 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T09:59:39.131 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:00:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T09:59:59.996+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T10:00:09.205 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:00:09.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:00:39.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:00:39.310 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:01:09.392 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:01:09.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:01:39.482 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:01:39.491 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:02:09.580 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:02:09.589 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:02:39.678 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:02:39.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:03:09.784 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:03:09.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:03:39.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:03:39.916 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:04:10.002 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:04:10.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:04:40.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:04:40.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:05:10.160 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:05:10.169 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:05:40.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:05:40.259 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:06:10.329 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:06:10.334 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:06:40.428 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:06:40.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:07:10.514 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:07:10.521 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:07:40.593 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:07:40.601 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:08:10.686 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:08:10.694 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:08:40.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:08:40.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:09:10.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:09:11.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:09:41.105 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:09:41.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:10:00.002 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T10:09:59.997+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T10:10:11.196 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:10:11.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:10:41.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:10:41.294 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:11:11.400 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:11:11.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:11:41.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:11:41.496 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:12:11.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:12:11.595 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:12:41.668 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:12:41.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:13:11.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:13:11.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:13:41.869 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:13:41.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:14:11.979 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:14:11.988 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:14:42.076 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:14:42.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:15:12.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:15:12.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:15:42.256 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:15:42.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:16:12.343 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:16:12.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:16:42.443 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:16:42.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:17:12.526 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:17:12.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:17:42.624 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:17:42.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:18:12.722 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:18:12.731 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:18:42.812 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:18:42.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:19:12.901 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:19:12.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:19:42.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:19:43.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:20:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T10:19:59.997+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T10:20:13.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:20:13.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:20:43.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:20:43.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:21:13.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:21:13.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:21:43.368 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:21:43.376 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:22:13.460 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:22:13.469 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:22:43.564 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:22:43.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:23:13.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:23:13.680 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:23:43.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:23:43.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:24:13.858 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:24:13.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:24:43.961 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:24:43.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:25:14.048 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:25:14.057 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:25:44.142 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:25:44.151 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:26:14.235 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:26:14.244 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:26:44.330 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:26:44.338 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:27:14.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:27:14.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:27:44.512 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:27:44.519 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:28:14.621 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:28:14.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:28:44.718 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:28:44.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:29:14.814 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:29:14.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:29:44.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:29:44.898 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:30:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T10:29:59.994+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T10:30:15.005 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:30:15.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:30:45.108 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:30:45.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:31:15.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:31:15.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:31:45.312 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:31:45.321 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:32:15.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:32:15.429 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:32:45.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:32:45.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:33:15.630 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:33:15.639 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:33:45.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:33:45.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:34:15.844 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:34:15.853 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:34:45.949 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:34:45.956 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:35:16.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:35:16.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:35:46.116 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:35:46.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:36:16.226 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:36:16.236 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:36:46.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:36:46.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:37:16.429 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:37:16.438 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:37:46.534 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:37:46.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:38:16.636 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:38:16.644 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:38:46.736 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:38:46.745 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:39:16.832 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:39:16.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:39:46.927 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:39:46.936 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:40:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T10:39:59.995+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T10:40:17.024 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:40:17.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:40:47.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:40:47.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:41:17.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:41:17.219 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:41:47.443 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:41:47.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:42:17.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:42:17.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:42:47.641 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:42:47.650 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:43:17.754 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:43:17.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:43:47.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:43:47.864 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:44:17.949 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:44:17.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:44:48.046 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:44:48.054 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:45:18.131 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:45:18.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:45:48.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:45:48.217 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:46:18.305 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:46:18.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:46:48.408 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:46:48.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:47:18.584 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:47:18.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:47:48.674 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:47:48.682 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:48:18.774 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:48:18.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:48:48.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:48:48.891 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:49:18.975 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:49:18.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:49:49.066 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:49:49.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:50:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T10:49:59.995+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T10:50:19.176 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:50:19.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:50:49.276 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:50:49.285 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:51:19.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:51:19.382 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:51:49.482 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:51:49.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:52:19.574 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:52:19.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:52:49.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:52:49.680 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:53:19.778 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:53:19.785 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:53:49.876 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:53:49.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:54:19.963 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:54:19.972 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:54:50.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:54:50.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:55:20.158 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:55:20.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:55:50.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:55:50.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:56:20.341 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:56:20.350 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:56:50.436 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:56:50.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:57:20.539 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:57:20.545 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:57:50.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:57:50.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:58:20.752 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:58:20.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:58:50.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:58:50.874 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:59:20.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:59:20.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:59:51.063 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T10:59:51.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:00:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T10:59:59.992+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T11:00:21.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:00:21.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:00:51.259 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:00:51.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:01:21.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:01:21.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:01:51.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:01:51.502 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:02:21.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:02:21.597 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:02:51.679 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:02:51.688 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:03:21.785 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:03:21.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:03:51.885 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:03:51.894 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:04:21.970 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:04:21.979 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:04:52.057 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:04:52.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:05:22.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:05:22.160 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:05:52.263 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:05:52.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:06:22.364 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:06:22.372 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:06:52.459 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:06:52.467 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:07:22.554 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:07:22.563 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:07:52.669 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:07:52.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:08:22.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:08:22.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:08:52.872 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:08:52.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:09:22.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:09:22.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:09:53.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:09:53.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:10:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T11:09:59.997+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T11:10:23.142 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:10:23.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:10:53.340 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:10:53.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:11:23.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:11:23.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:11:53.527 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:11:53.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:12:23.621 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:12:23.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:12:53.720 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:12:53.730 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:13:23.831 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:13:23.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:13:53.934 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:13:53.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:14:24.021 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:14:24.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:14:54.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:14:54.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:15:24.217 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:15:24.225 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:15:54.346 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:15:54.354 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:16:24.443 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:16:24.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:16:54.546 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:16:54.554 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:17:24.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:17:24.644 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:17:54.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:17:54.719 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:18:24.810 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:18:24.818 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:18:54.896 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:18:54.905 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:19:24.987 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:19:24.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:19:55.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:19:55.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:20:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T11:19:59.997+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T11:20:25.182 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:20:25.192 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:20:55.280 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:20:55.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:21:25.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:21:25.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:21:55.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:21:55.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:22:25.545 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:22:25.554 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:22:55.647 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:22:55.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:23:25.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:23:25.748 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:23:55.814 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:23:55.823 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:24:25.908 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:24:25.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:24:56.016 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:24:56.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:25:26.115 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:25:26.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:25:56.204 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:25:56.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:26:26.303 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:26:26.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:26:56.408 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:26:56.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:27:26.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:27:26.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:27:56.671 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:27:56.680 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:28:26.778 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:28:26.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:28:56.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:28:56.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:29:26.966 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:29:26.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:29:57.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:29:57.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:30:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T11:29:59.998+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T11:30:27.252 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:30:27.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:30:57.343 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:30:57.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:31:27.451 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:31:27.460 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:31:57.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:31:57.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:32:27.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:32:27.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:32:57.751 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:32:57.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:33:27.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:33:27.857 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:33:57.931 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:33:57.940 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:34:28.035 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:34:28.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:34:58.112 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:34:58.122 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:35:28.193 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:35:28.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:35:58.289 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:35:58.297 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:36:28.379 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:36:28.388 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:36:58.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:36:58.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:37:28.570 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:37:28.579 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:37:58.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:37:58.679 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:38:28.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:38:28.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:38:58.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:38:58.864 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:39:28.946 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:39:28.953 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:39:59.038 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:39:59.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:40:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T11:39:59.998+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T11:40:29.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:40:29.164 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:40:59.264 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:40:59.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:41:29.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:41:29.368 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:41:59.465 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:41:59.474 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:42:29.569 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:42:29.577 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:42:59.661 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:42:59.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:43:29.763 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:43:29.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:43:59.884 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:43:59.890 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:44:29.978 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:44:29.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:45:00.088 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:45:00.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:45:30.171 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:45:30.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:46:00.260 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:46:00.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:46:30.349 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:46:30.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:47:00.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:47:00.469 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:47:30.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:47:30.549 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:48:00.642 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:48:00.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:48:30.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:48:30.748 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:49:00.834 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:49:00.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:49:30.928 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:49:30.936 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:50:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T11:49:59.995+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T11:50:01.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:50:01.057 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:50:31.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:50:31.161 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:51:01.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:51:01.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:51:31.351 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:51:31.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:52:01.475 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:52:01.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:52:31.582 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:52:31.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:53:01.680 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:53:01.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:53:31.785 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:53:31.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:54:01.886 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:54:01.894 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:54:31.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:54:31.997 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:55:02.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:55:02.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:55:32.175 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:55:32.184 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:56:02.276 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:56:02.284 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:56:32.413 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:56:32.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:57:02.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:57:02.544 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:57:32.628 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:57:32.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:58:02.757 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:58:02.766 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:58:32.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:58:32.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:59:02.957 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:59:02.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:59:33.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T11:59:33.057 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:00:00.002 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T11:59:59.996+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T12:00:03.171 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:00:03.179 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:00:33.259 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:00:33.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:01:03.357 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:01:03.364 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:01:33.439 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:01:33.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:02:03.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:02:03.545 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:02:33.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:02:33.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:03:03.731 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:03:03.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:03:33.831 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:03:33.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:04:03.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:04:03.937 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:04:34.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:04:34.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:05:04.121 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:05:04.130 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:05:34.217 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:05:34.226 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:06:04.318 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:06:04.326 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:06:34.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:06:34.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:07:04.508 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:07:04.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:07:34.605 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:07:34.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:08:04.706 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:08:04.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:08:34.807 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:08:34.816 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:09:04.895 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:09:04.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:09:34.991 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:09:35.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:10:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T12:09:59.996+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T12:10:05.086 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:10:05.095 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:10:35.187 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:10:35.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:11:05.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:11:05.281 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:11:35.358 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:11:35.366 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:12:05.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:12:05.465 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:12:35.551 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:12:35.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:13:05.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:13:05.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:13:35.838 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:13:35.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:14:05.933 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:14:05.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:14:36.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:14:36.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:15:06.116 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:15:06.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:15:36.216 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:15:36.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:16:06.314 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:16:06.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:16:36.424 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:16:36.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:17:06.508 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:17:06.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:17:36.608 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:17:36.617 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:18:06.692 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:18:06.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:18:36.791 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:18:36.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:19:06.889 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:19:06.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:19:36.990 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:19:36.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:20:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T12:19:59.993+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T12:20:07.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:20:07.099 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:20:37.185 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:20:37.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:21:07.298 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:21:07.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:21:37.387 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:21:37.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:22:07.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:22:07.502 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:22:37.587 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:22:37.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:23:07.678 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:23:07.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:23:37.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:23:37.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:24:07.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:24:07.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:24:37.964 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:24:37.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:25:08.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:25:08.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:25:38.148 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:25:38.157 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:26:08.249 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:26:08.258 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:26:38.344 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:26:38.353 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:27:08.451 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:27:08.460 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:27:38.545 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:27:38.554 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:28:08.650 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:28:08.659 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:28:38.758 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:28:38.767 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:29:08.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:29:08.862 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:29:38.938 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:29:38.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:30:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T12:29:59.997+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T12:30:09.029 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:30:09.036 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:30:39.115 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:30:39.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:31:09.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:31:09.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:31:39.306 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:31:39.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:32:09.405 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:32:09.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:32:39.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:32:39.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:33:09.697 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:33:09.706 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:33:39.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:33:39.803 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:34:09.894 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:34:09.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:34:40.009 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:34:40.017 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:35:10.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:35:10.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:35:40.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:35:40.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:36:10.289 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:36:10.294 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:36:40.399 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:36:40.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:37:10.482 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:37:10.491 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:37:40.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:37:40.582 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:38:10.669 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:38:10.679 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:38:40.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:38:40.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:39:10.879 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:39:10.888 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:39:40.972 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:39:40.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:40:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T12:39:59.994+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T12:40:11.067 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:40:11.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:40:41.163 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:40:41.172 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:41:11.251 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:41:11.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:41:41.342 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:41:41.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:42:11.439 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:42:11.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:42:41.539 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:42:41.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:43:11.633 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:43:11.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:43:41.722 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:43:41.730 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:44:11.813 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:44:11.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:44:41.902 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:44:41.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:45:12.013 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:45:12.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:45:42.098 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:45:42.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:46:12.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:46:12.427 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:46:42.498 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:46:42.507 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:47:12.597 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:47:12.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:47:42.693 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:47:42.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:48:12.787 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:48:12.796 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:48:42.926 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:48:42.934 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:49:13.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:49:13.064 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:49:43.146 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:49:43.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:50:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T12:49:59.994+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T12:50:13.241 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:50:13.250 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:50:43.340 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:50:43.349 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:51:13.431 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:51:13.439 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:51:43.544 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:51:43.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:52:13.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:52:13.649 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:52:43.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:52:43.731 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:53:13.823 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:53:13.832 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:53:43.918 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:53:43.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:54:14.015 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:54:14.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:54:44.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:54:44.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:55:14.199 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:55:14.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:55:44.293 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:55:44.302 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:56:14.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:56:14.393 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:56:44.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:56:44.499 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:57:14.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:57:14.597 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:57:44.679 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:57:44.688 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:58:14.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:58:14.785 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:58:44.965 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:58:44.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:59:15.067 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:59:15.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:59:45.164 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T12:59:45.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:00:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T12:59:59.995+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T13:00:15.243 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:00:15.251 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:00:45.333 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:00:45.342 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:01:15.411 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:01:15.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:01:45.519 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:01:45.527 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:02:15.611 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:02:15.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:02:45.699 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:02:45.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:03:15.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:03:15.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:03:45.893 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:03:45.902 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:04:15.998 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:04:16.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:04:46.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:04:46.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:05:16.206 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:05:16.215 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:05:46.296 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:05:46.305 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:06:16.392 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:06:16.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:06:46.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:06:46.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:07:16.572 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:07:16.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:07:46.664 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:07:46.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:08:16.769 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:08:16.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:08:46.868 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:08:46.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:09:16.955 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:09:16.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:09:47.061 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:09:47.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:10:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T13:09:59.995+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T13:10:17.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:10:17.166 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:10:47.248 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:10:47.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:11:17.337 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:11:17.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:11:47.433 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:11:47.442 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:12:17.531 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:12:17.540 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:12:47.620 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:12:47.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:13:17.709 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:13:17.718 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:13:47.807 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:13:47.816 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:14:17.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:14:17.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:14:47.993 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:14:48.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:15:18.147 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:15:18.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:15:48.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:15:48.275 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:16:18.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:16:18.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:16:48.474 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:16:48.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:17:18.556 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:17:18.564 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:17:48.654 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:17:48.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:18:18.749 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:18:18.758 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:18:48.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:18:48.871 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:19:18.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:19:18.960 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:19:49.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:19:49.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:20:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T13:19:59.996+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T13:20:19.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:20:19.160 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:20:49.267 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:20:49.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:21:19.414 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:21:19.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:21:49.503 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:21:49.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:22:19.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:22:19.611 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:22:49.695 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:22:49.703 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:23:19.805 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:23:19.814 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:23:49.903 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:23:49.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:24:20.001 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:24:20.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:24:50.097 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:24:50.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:25:20.188 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:25:20.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:25:50.261 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:25:50.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:26:20.342 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:26:20.350 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:26:50.426 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:26:50.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:27:20.548 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:27:20.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:27:50.647 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:27:50.654 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:28:20.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:28:20.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:28:50.852 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:28:50.859 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:29:20.933 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:29:20.941 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:29:51.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:29:51.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:30:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T13:29:59.993+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T13:30:21.130 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:30:21.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:30:51.234 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:30:51.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:31:21.331 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:31:21.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:31:51.424 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:31:51.433 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:32:21.521 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:32:21.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:32:51.619 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:32:51.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:33:21.707 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:33:21.716 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:33:51.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:33:51.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:34:21.895 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:34:21.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:34:52.107 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:34:52.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:35:22.187 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:35:22.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:35:52.277 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:35:52.286 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:36:22.392 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:36:22.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:36:52.480 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:36:52.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:37:22.582 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:37:22.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:37:52.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:37:52.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:38:22.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:38:22.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:38:52.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:38:52.929 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:39:23.014 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:39:23.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:39:53.097 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:39:53.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:40:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T13:39:59.997+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T13:40:23.193 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:40:23.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:40:53.291 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:40:53.296 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:41:23.390 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:41:23.398 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:41:53.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:41:53.511 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:42:23.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:42:23.610 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:42:53.694 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:42:53.702 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:43:23.797 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:43:23.805 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:43:53.899 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:43:53.908 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:44:23.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:44:24.008 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:44:54.099 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:44:54.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:45:24.188 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:45:24.197 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:45:54.278 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:45:54.287 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:46:24.380 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:46:24.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:46:54.472 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:46:54.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:47:24.556 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:47:24.565 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:47:54.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:47:54.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:48:24.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:48:24.752 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:48:54.843 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:48:54.851 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:49:24.939 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:49:24.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:49:55.021 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:49:55.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:50:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T13:49:59.998+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T13:50:25.122 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:50:25.131 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:50:55.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:50:55.221 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:51:25.300 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:51:25.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:51:55.378 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:51:55.388 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:52:25.485 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:52:25.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:52:55.570 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:52:55.578 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:53:25.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:53:25.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:53:55.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:53:55.769 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:54:25.877 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:54:25.886 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:54:55.987 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:54:55.995 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:55:26.099 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:55:26.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:55:56.182 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:55:56.190 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:56:26.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:56:26.304 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:56:56.378 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:56:56.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:57:26.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:57:26.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:57:56.568 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:57:56.576 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:58:26.668 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:58:26.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:58:56.764 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:58:56.773 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:59:26.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:59:26.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:59:56.952 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T13:59:56.961 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:00:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T13:59:59.998+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T14:00:27.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:00:27.049 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:00:57.117 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:00:57.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:01:27.216 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:01:27.225 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:01:57.415 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:01:57.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:02:27.503 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:02:27.512 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:02:57.598 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:02:57.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:03:27.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:03:27.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:03:57.792 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:03:57.801 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:04:27.889 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:04:27.898 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:04:57.970 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:04:57.980 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:05:28.073 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:05:28.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:05:58.162 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:05:58.172 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:06:28.260 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:06:28.274 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:06:58.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:06:58.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:07:28.449 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:07:28.458 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:07:58.527 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:07:58.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:08:28.641 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:08:28.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:08:58.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:08:58.753 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:09:28.842 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:09:28.853 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:09:58.936 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:09:58.948 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:10:00.006 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T14:09:59.999+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T14:10:29.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:10:29.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:10:59.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:10:59.147 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:11:29.226 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:11:29.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:11:59.332 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:11:59.342 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:12:29.413 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:12:29.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:12:59.507 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:12:59.516 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:13:29.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:13:29.597 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:13:59.676 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:13:59.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:14:29.777 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:14:29.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:14:59.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:14:59.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:15:29.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:15:29.984 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:16:00.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:16:00.074 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:16:30.164 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:16:30.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:17:00.267 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:17:00.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:17:30.358 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:17:30.366 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:18:00.519 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:18:00.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:18:30.606 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:18:30.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:19:00.698 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:19:00.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:19:30.797 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:19:30.805 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:20:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T14:20:00.000+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T14:20:00.887 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:20:00.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:20:30.983 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:20:30.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:21:01.076 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:21:01.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:21:31.172 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:21:31.181 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:22:01.367 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:22:01.376 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:22:31.461 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:22:31.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:23:01.559 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:23:01.568 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:23:31.658 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:23:31.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:24:01.757 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:24:01.766 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:24:31.852 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:24:31.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:25:01.936 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:25:01.945 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:25:32.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:25:32.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:26:02.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:26:02.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:26:32.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:26:32.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:27:02.298 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:27:02.307 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:27:32.389 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:27:32.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:28:02.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:28:02.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:28:32.576 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:28:32.586 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:29:02.680 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:29:02.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:29:32.777 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:29:32.787 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:30:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T14:29:59.992+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T14:30:02.886 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:30:02.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:30:32.982 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:30:32.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:31:03.064 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:31:03.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:31:33.162 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:31:33.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:32:03.249 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:32:03.253 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:32:33.337 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:32:33.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:33:03.428 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:33:03.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:33:33.514 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:33:33.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:34:03.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:34:03.611 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:34:33.701 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:34:33.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:35:03.792 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:35:03.801 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:35:33.897 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:35:33.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:36:03.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:36:04.003 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:36:34.100 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:36:34.106 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:37:04.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:37:04.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:37:34.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:37:34.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:38:04.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:38:04.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:38:34.453 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:38:34.461 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:39:04.564 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:39:04.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:39:34.669 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:39:34.678 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:40:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T14:39:59.993+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T14:40:04.755 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:40:04.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:40:34.854 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:40:34.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:41:04.935 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:41:04.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:41:35.016 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:41:35.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:42:05.102 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:42:05.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:42:35.198 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:42:35.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:43:05.342 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:43:05.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:43:35.424 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:43:35.433 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:44:05.522 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:44:05.531 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:44:35.619 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:44:35.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:45:05.703 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:45:05.712 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:45:35.784 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:45:35.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:46:05.873 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:46:05.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:46:35.967 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:46:35.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:47:06.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:47:06.062 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:47:36.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:47:36.161 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:48:06.241 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:48:06.249 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:48:36.338 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:48:36.347 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:49:06.420 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:49:06.429 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:49:36.517 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:49:36.527 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:50:00.002 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T14:49:59.993+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T14:50:06.607 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:50:06.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:50:36.687 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:50:36.696 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:51:06.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:51:06.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:51:36.874 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:51:36.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:52:06.957 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:52:06.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:52:37.047 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:52:37.056 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:53:07.146 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:53:07.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:53:37.261 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:53:37.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:54:07.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:54:07.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:54:37.462 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:54:37.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:55:07.549 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:55:07.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:55:37.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:55:37.666 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:56:07.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:56:07.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:56:37.860 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:56:37.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:57:07.947 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:57:07.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:57:38.047 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:57:38.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:58:08.131 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:58:08.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:58:38.232 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:58:38.241 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:59:08.333 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:59:08.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:59:38.441 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T14:59:38.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:00:00.004 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T14:59:59.998+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T15:00:08.529 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:00:08.538 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:00:38.630 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:00:38.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:01:08.719 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:01:08.728 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:01:38.832 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:01:38.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:02:08.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:02:08.969 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:02:39.141 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:02:39.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:03:09.226 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:03:09.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:03:39.312 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:03:39.321 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:04:09.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:04:09.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:04:39.516 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:04:39.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:05:09.619 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:05:09.627 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:05:39.726 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:05:39.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:06:09.813 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:06:09.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:06:39.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:06:39.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:07:09.998 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:07:10.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:07:40.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:07:40.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:08:10.186 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:08:10.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:08:40.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:08:40.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:09:10.367 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:09:10.376 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:09:40.452 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:09:40.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:10:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T15:09:59.994+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T15:10:10.547 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:10:10.554 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:10:40.637 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:10:40.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:11:10.722 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:11:10.731 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:11:40.820 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:11:40.828 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:12:10.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:12:10.926 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:12:41.013 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:12:41.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:13:11.103 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:13:11.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:13:41.200 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:13:41.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:14:11.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:14:11.310 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:14:41.404 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:14:41.414 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:15:11.493 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:15:11.502 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:15:41.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:15:41.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:16:11.687 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:16:11.697 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:16:41.790 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:16:41.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:17:11.875 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:17:11.884 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:17:41.979 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:17:41.989 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:18:12.070 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:18:12.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:18:42.152 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:18:42.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:19:12.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:19:12.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:19:42.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:19:42.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:20:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T15:19:59.999+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T15:20:12.448 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:20:12.456 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:20:42.554 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:20:42.563 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:21:12.644 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:21:12.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:21:42.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:21:42.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:22:12.819 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:22:12.828 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:22:42.923 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:22:42.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:23:13.000 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:23:13.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:23:43.103 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:23:43.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:24:13.201 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:24:13.209 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:24:43.300 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:24:43.307 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:25:13.391 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:25:13.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:25:43.474 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:25:43.483 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:26:13.564 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:26:13.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:26:43.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:26:43.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:27:13.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:27:13.754 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:27:43.846 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:27:43.855 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:28:13.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:28:13.961 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:28:44.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:28:44.057 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:29:14.149 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:29:14.158 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:29:44.302 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:29:44.310 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:30:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T15:29:59.999+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T15:30:14.390 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:30:14.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:30:44.480 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:30:44.489 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:31:14.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:31:14.571 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:31:44.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:31:44.728 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:32:14.812 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:32:14.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:32:44.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:32:44.927 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:33:15.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:33:15.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:33:45.117 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:33:45.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:34:15.222 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:34:15.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:34:45.317 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:34:45.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:35:15.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:35:15.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:35:45.491 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:35:45.500 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:36:15.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:36:15.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:36:45.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:36:45.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:37:15.750 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:37:15.759 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:37:45.841 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:37:45.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:38:15.932 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:38:15.942 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:38:46.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:38:46.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:39:16.124 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:39:16.133 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:39:46.226 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:39:46.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:40:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T15:39:59.996+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T15:40:16.309 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:40:16.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:40:46.391 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:40:46.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:41:16.474 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:41:16.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:41:46.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:41:46.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:42:16.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:42:16.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:42:46.757 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:42:46.766 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:43:16.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:43:16.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:43:47.005 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:43:47.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:44:17.097 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:44:17.106 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:44:47.200 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:44:47.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:45:17.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:45:17.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:45:47.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:45:47.395 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:46:17.481 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:46:17.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:46:47.583 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:46:47.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:47:17.671 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:47:17.680 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:47:47.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:47:47.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:48:17.836 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:48:17.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:48:47.933 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:48:47.942 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:49:18.037 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:49:18.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:49:48.138 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:49:48.147 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:50:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T15:49:59.996+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T15:50:18.255 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:50:18.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:50:48.358 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:50:48.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:51:18.445 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:51:18.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:51:48.546 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:51:48.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:52:18.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:52:18.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:52:48.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:52:48.748 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:53:18.842 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:53:18.851 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:53:48.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:53:48.930 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:54:19.020 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:54:19.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:54:49.112 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:54:49.121 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:55:19.196 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:55:19.204 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:55:49.299 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:55:49.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:56:19.385 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:56:19.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:56:49.476 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:56:49.486 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:57:19.565 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:57:19.574 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:57:49.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:57:49.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:58:19.749 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:58:19.758 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:58:49.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:58:49.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:59:19.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:59:19.961 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:59:50.037 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T15:59:50.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:00:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T15:59:59.993+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T16:00:20.137 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:00:20.146 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:00:50.245 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:00:50.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:01:20.323 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:01:20.332 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:01:50.411 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:01:50.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:02:20.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:02:20.496 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:02:50.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:02:50.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:03:20.681 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:03:20.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:03:50.774 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:03:50.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:04:20.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:04:20.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:04:50.962 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:04:50.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:05:21.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:05:21.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:05:51.152 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:05:51.160 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:06:21.307 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:06:21.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:06:51.405 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:06:51.413 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:07:21.481 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:07:21.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:07:51.587 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:07:51.595 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:08:21.675 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:08:21.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:08:51.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:08:51.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:09:21.872 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:09:21.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:09:51.970 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:09:51.979 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:10:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T16:09:59.994+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T16:10:22.054 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:10:22.062 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:10:52.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:10:52.143 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:11:22.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:11:22.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:11:52.302 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:11:52.311 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:12:22.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:12:22.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:12:52.496 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:12:52.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:13:22.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:13:22.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:13:52.685 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:13:52.694 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:14:22.791 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:14:22.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:14:52.889 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:14:52.896 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:15:22.979 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:15:22.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:15:53.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:15:53.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:16:23.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:16:23.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:16:53.266 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:16:53.274 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:17:23.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:17:23.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:17:53.469 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:17:53.478 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:18:23.561 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:18:23.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:18:53.675 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:18:53.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:19:23.774 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:19:23.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:19:53.870 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:19:53.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:20:00.001 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T16:19:59.998+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T16:20:23.954 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:20:23.962 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:20:54.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:20:54.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:21:24.142 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:21:24.152 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:21:54.239 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:21:54.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:22:24.320 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:22:24.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:22:54.420 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:22:54.429 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:23:24.512 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:23:24.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:23:54.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:23:54.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:24:24.714 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:24:24.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:24:54.811 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:24:54.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:25:24.896 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:25:24.905 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:25:55.011 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:25:55.020 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:26:25.102 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:26:25.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:26:55.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:26:55.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:27:25.289 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:27:25.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:27:55.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:27:55.383 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:28:25.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:28:25.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:28:55.542 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:28:55.551 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:29:25.637 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:29:25.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:29:55.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:29:55.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:30:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T16:29:59.995+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T16:30:25.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:30:25.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:30:55.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:30:55.918 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:31:26.001 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:31:26.011 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:31:56.099 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:31:56.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:32:26.186 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:32:26.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:32:56.282 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:32:56.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:33:26.363 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:33:26.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:33:56.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:33:56.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:34:26.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:34:26.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:34:56.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:34:56.654 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:35:26.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:35:26.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:35:56.836 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:35:56.845 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:36:26.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:36:26.938 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:36:57.033 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:36:57.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:37:27.145 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:37:27.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:37:57.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:37:57.253 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:38:27.326 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:38:27.334 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:38:57.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:38:57.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:39:27.515 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:39:27.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:39:57.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:39:57.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:40:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T16:39:59.996+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T16:40:27.703 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:40:27.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:40:57.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:40:57.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:41:27.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:41:27.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:41:57.966 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:41:57.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:42:28.054 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:42:28.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:42:58.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:42:58.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:43:28.243 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:43:28.253 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:43:58.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:43:58.357 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:44:28.444 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:44:28.453 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:44:58.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:44:58.547 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:45:28.644 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:45:28.653 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:45:58.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:45:58.743 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:46:28.823 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:46:28.831 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:46:58.922 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:46:58.930 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:47:29.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:47:29.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:47:59.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:47:59.119 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:48:29.198 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:48:29.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:48:59.298 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:48:59.308 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:49:29.414 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:49:29.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:49:59.516 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:49:59.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:50:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T16:49:59.997+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T16:50:29.610 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:50:29.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:50:59.706 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:50:59.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:51:29.789 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:51:29.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:51:59.879 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:51:59.889 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:52:29.968 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:52:29.976 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:53:00.067 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:53:00.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:53:30.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:53:30.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:54:00.255 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:54:00.262 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:54:30.353 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:54:30.361 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:55:00.435 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:55:00.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:55:30.516 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:55:30.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:56:00.613 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:56:00.623 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:56:30.697 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:56:30.706 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:57:00.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:57:00.805 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:57:30.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:57:30.893 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:58:00.963 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:58:00.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:58:31.048 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:58:31.057 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:59:01.144 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:59:01.153 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:59:31.235 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T16:59:31.244 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:00:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T16:59:59.993+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T17:00:01.330 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:00:01.339 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:00:31.414 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:00:31.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:01:01.512 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:01:01.519 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:01:31.593 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:01:31.602 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:02:01.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:02:01.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:02:31.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:02:31.785 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:03:01.871 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:03:01.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:03:31.970 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:03:31.978 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:04:02.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:04:02.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:04:32.155 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:04:32.164 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:05:02.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:05:02.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:05:32.328 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:05:32.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:06:02.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:06:02.427 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:06:32.506 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:06:32.516 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:07:02.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:07:02.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:07:32.687 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:07:32.697 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:08:02.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:08:02.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:08:32.874 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:08:32.883 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:09:02.982 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:09:02.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:09:33.071 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:09:33.080 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:10:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T17:09:59.994+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T17:10:03.154 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:10:03.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:10:33.233 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:10:33.241 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:11:03.332 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:11:03.341 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:11:33.422 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:11:33.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:12:03.529 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:12:03.539 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:12:33.621 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:12:33.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:13:03.719 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:13:03.731 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:13:33.825 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:13:33.836 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:14:03.926 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:14:03.937 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:14:34.031 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:14:34.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:15:04.130 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:15:04.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:15:34.220 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:15:34.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:16:04.318 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:16:04.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:16:34.408 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:16:34.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:17:04.503 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:17:04.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:17:34.587 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:17:34.597 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:18:04.686 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:18:04.694 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:18:34.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:18:34.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:19:04.873 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:19:04.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:19:34.962 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:19:34.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:20:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T17:19:59.995+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T17:20:05.062 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:20:05.069 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:20:35.140 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:20:35.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:21:05.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:21:05.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:21:35.329 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:21:35.338 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:22:05.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:22:05.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:22:35.509 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:22:35.518 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:23:05.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:23:05.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:23:35.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:23:35.680 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:24:05.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:24:05.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:24:35.869 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:24:35.874 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:25:05.958 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:25:05.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:25:36.048 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:25:36.058 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:26:06.154 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:26:06.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:26:36.243 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:26:36.253 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:27:06.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:27:06.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:27:36.407 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:27:36.417 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:28:06.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:28:06.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:28:36.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:28:36.609 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:29:06.692 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:29:06.701 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:29:36.792 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:29:36.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:30:00.000 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T17:29:59.996+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T17:30:06.897 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:30:06.906 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:30:36.986 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:30:36.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:31:07.102 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:31:07.110 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:31:37.192 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:31:37.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:32:07.291 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:32:07.301 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:32:37.380 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:32:37.388 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:33:07.475 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:33:07.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:33:37.563 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:33:37.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:34:07.647 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:34:07.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:34:37.755 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:34:37.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:35:07.862 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:35:07.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:35:37.951 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:35:37.962 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:36:08.058 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:36:08.068 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:36:38.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:36:38.160 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:37:08.252 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:37:08.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:37:38.336 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:37:38.361 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:38:08.477 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:38:08.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:38:38.564 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:38:38.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:39:08.661 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:39:08.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:39:38.754 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:39:38.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:39:59.999 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T17:39:59.997+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T17:40:08.860 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:40:08.869 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:40:38.959 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:40:38.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:41:09.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:41:09.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:41:39.165 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:41:39.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:42:09.260 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:42:09.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:42:39.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:42:39.361 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:43:09.458 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:43:09.465 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:43:39.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:43:39.546 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:44:09.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:44:09.644 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:44:39.734 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:44:39.742 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:45:09.831 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:45:09.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:45:39.923 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:45:39.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:46:10.009 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:46:10.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:46:40.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:46:40.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:47:10.197 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:47:10.205 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:47:40.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:47:40.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:48:10.367 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:48:10.375 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:48:40.456 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:48:40.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:49:10.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:49:10.546 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:49:40.643 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:49:40.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:49:59.999 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T17:49:59.994+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T17:50:10.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:50:10.754 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:50:40.826 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:50:40.834 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:51:10.924 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:51:10.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:51:41.014 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:51:41.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:52:11.113 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:52:11.122 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:52:41.201 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:52:41.210 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:53:11.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:53:11.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:53:41.387 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:53:41.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:54:11.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:54:11.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:54:41.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:54:41.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:55:11.674 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:55:11.683 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:55:41.755 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:55:41.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:56:11.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:56:11.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:56:41.935 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:56:41.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:57:12.033 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:57:12.041 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:57:42.146 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:57:42.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:58:12.245 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:58:12.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:58:42.332 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:58:42.341 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:59:12.435 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:59:12.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:59:42.533 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:59:42.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T17:59:59.998 INFO:tasks.ceph.mon.a.smithi003.stderr:2023-03-03T17:59:59.995+0000 7f07393f9700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2023-03-03T18:00:12.631 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:00:12.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:00:42.719 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:00:42.728 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:01:12.801 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:01:12.809 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:01:42.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:01:42.899 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:02:12.995 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:02:13.003 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:02:43.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:02:43.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:03:13.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:03:13.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:03:43.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:03:43.266 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:04:13.356 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:04:13.366 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:04:43.455 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:04:43.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:05:13.553 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:05:13.562 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:05:43.731 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:05:43.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:06:13.816 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:06:13.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:06:43.906 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:06:43.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:07:13.996 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:07:14.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:07:44.085 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:07:44.094 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:08:14.184 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:08:14.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:08:44.271 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:08:44.280 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-03T18:08:50.439 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2023-03-03T18:08:50.440 DEBUG:teuthology.task.console_log:Killing console logger for smithi003 2023-03-03T18:08:50.441 DEBUG:teuthology.task.console_log:Killing console logger for smithi032 2023-03-03T18:08:50.441 DEBUG:teuthology.exit:Finished running handlers